Vision system for vehicle

ABSTRACT

A vision system for a vehicle includes a forward viewing camera. Responsive at least in part to processing by an image processor of image data captured by the forward viewing camera, at least one road characteristic of a road along which the vehicle is traveling is detected. Responsive at least in part to processing by the image processor of image data captured by the forward viewing camera, other vehicles exterior the equipped vehicle are detected. Responsive at least in part to processing by the image processor of image data captured by the forward viewing camera, road curvature of the road along which the vehicle is traveling is determined. Data derived from image data captured by the forward viewing camera may be wirelessly communicated from the vehicle to a remote data receiver located remote from the vehicle.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/463,296, filed Mar. 20, 2017, now U.S. Pat. No. 9,736,435,which is a continuation of U.S. patent application Ser. No. 15/249,557,filed Aug. 29, 2016, now U.S. Pat. No. 9,609,289, which is acontinuation of U.S. patent application Ser. No. 14/942,089, filed Nov.16, 2015, now U.S. Pat. No. 9,428,192, which is a continuation of U.S.patent application Ser. No. 14/678,146, filed Apr. 3, 2015, now U.S.Pat. No. 9,191,634, which is a continuation of U.S. patent applicationSer. No. 14/467,296, filed Aug. 25, 2014, now U.S. Pat. No. 9,008,369,which is a continuation of U.S. patent application Ser. No. 14/082,577,filed Nov. 18, 2013, now U.S. Pat. No. 8,818,042, which is acontinuation of U.S. patent application Ser. No. 13/689,796, filed Nov.30, 2012, now U.S. Pat. No. 8,593,521, which is a continuation of U.S.patent application Ser. No. 13/335,125, filed Dec. 22, 2011, now U.S.Pat. No. 8,325,986, which is a continuation of U.S. patent applicationSer. No. 13/107,318, filed May 13, 2011, now U.S. Pat. No. 8,090,153,which is a continuation of U.S. patent application Ser. No. 12/979,499,filed Dec. 28, 2010, now U.S. Pat. No. 7,949,152, which is acontinuation of U.S. patent application Ser. No. 12/856,737, filed Aug.16, 2010, now U.S. Pat. No. 7,873,187, which is a continuation of U.S.patent application Ser. No. 12/606,476, filed Oct. 27, 2009, now U.S.Pat. No. 7,792,329, which is a continuation of U.S. patent applicationSer. No. 12/429,605, filed Apr. 24, 2009, now U.S. Pat. No. 7,616,781,which is a continuation of U.S. patent application Ser. No. 11/105,757,filed Apr. 14, 2005, now U.S. Pat. No. 7,526,103, which claims benefitof U.S. provisional applications, Ser. No. 60/644,903, filed Jan. 19,2005; Ser. No. 60/642,227, filed Jan. 7, 2005; Ser. No. 60/607,963,filed Sep. 8, 2004; and Ser. No. 60/562,480, filed Apr. 15, 2004, whichare hereby incorporated herein by reference in their entireties.

FIELD OF THE INVENTION

The present invention relates generally to an imaging system for avehicle and, more particularly, to an imaging system that may displayinformation to a driver of the vehicle and/or control an accessory inresponse to images captured by a camera or image capture device.

BACKGROUND OF THE INVENTION

It is known to provide an image capture device at a vehicle forcapturing images of the scene occurring exteriorly of the vehicle, suchas forwardly or rearwardly or sidewardly of the vehicle. The capturedimages may be processed by a processing system and the system maycontrol the headlamps of the vehicle or may provide an image display tothe driver of the vehicle or may provide other information or signals,depending on the particular application of the imaging system.

SUMMARY OF THE INVENTION

The present invention provides an imaging system for a vehicle that isoperable to identify and read traffic control signage as the vehicletravels along a road. The system may provide an information displayand/or alert to a driver of the vehicle in response to the imagescaptured by a camera or imaging device of the vehicle. The imagingsystem of the present invention may also process captured images andcontrol one or more accessories in response to such processing. Forexample, the imaging system of the present invention may control theheadlamps or may adjust or control the direction of the headlamps inresponse to such image processing.

According to an aspect of the present invention, an imaging system for avehicle includes an imaging device, a display device and an imageprocessor. The imaging device has a field of view exteriorly and forwardof the vehicle in its direction of travel and captures imagesrepresentative of the exterior scene. The image processor processes thecaptured images and determines whether the captured image encompasses animage of a traffic control sign. If the image processor determines thatthe captured image encompasses a traffic control sign of interest, theimage processor determines the numerals, characters or other informationon the face of the sign. The image processor may control the displaydevice in response to the determined characters or information and inresponse to a vehicle speed input that is indicative of the speed thatthe vehicle is traveling. The display device thus may displayinformation to a driver of the vehicle in response to an output of theimage processor and/or may generate at least one of a visible, audibleor tactile/haptic signal to alert the driver that he or she has entereda different speed zone. Most preferably, the display information and/oralert differentiates and distinguishes from and is characteristic of anallowed increase in driving speed from one zone to another and adecrease in driving speed from one zone to another, whereby the driveris informed as to whether it is allowable to drive faster or is requiredto drive slower.

Preferably, the imaging device and the associated image processor arelocated within the interior cabin of the vehicle with a field of viewthrough the vehicle windshield and, most preferably, the image processoris located at an interior rearview mirror assembly or at a windshieldelectronic module located at a central upper windshield location.Preferably, the imaging system can be multi-tasking, and thus may bepart of or associated with other vehicle accessories or systems or mayaccess or share components or circuitry of other vehicle accessories orsystems. For example, the image processor may preferably derive furtherinformation from the processed captured images, such as a determinationof location, intensity and type of oncoming headlamps or leadingtaillights being approached by the vehicle, rain or fog or the likepresent and detected within the forward field of view, a presence ofobstacles or objects or vehicles in the forward field of view and/or thelike, such as in connection with a headlamp control system, aprecipitation sensor, an adaptive cruise control system, a lanedeparture warning system, a traffic lane control system and/or the like.

For example, the image processor may determine that a speed limit signis within the captured image by analyzing the size, shape and locationof a detected object. The image processor may then determine orrecognize the characters or numbers or information on the face of thespeed limit sign to determine the speed limit in the area through whichthe vehicle is traveling. The display device may display information tothe driver of the vehicle in response to the determined characters andthe vehicle speed. For example, if the vehicle speed is substantiallygreater than the posted and determined speed limit, the display devicemay display information to that effect or may provide a warning or alertsignal to alert the driver of the excessive speed that the vehicle istraveling.

According to another aspect of the present invention, an imaging systemfor a vehicle includes an imaging device and an image processor. Theimaging device has a field of view exteriorly and forward of the vehiclein its direction of travel. The imaging device is operable to captureimages representative of the exterior scene. The image processor isoperable to process the captured images in accordance with an algorithm.The algorithm comprises a sign recognition routine and a characterrecognition routine.

The algorithm may pass to the character recognition routine after thesign recognition routine. The image processor may process the capturedimage to determine whether the captured image encompasses an image of atraffic control sign of interest when in the sign recognition routine.The image processor may process the captured image to determine what thecharacters on the face of the sign represent when in the characterrecognition routine. The algorithm may proceed to the characterrecognition routine in response to the image processor determining thatthe captured image encompasses an image of a traffic control sign ofinterest.

The imaging system may include at least one of a visible alert, anaudible alert and a tactile alert to a driver of the vehicle in responseto an output of the image processor. The visible alert may displayinformation indicative of at least one of the vehicle speed, a postedspeed limit and a difference between the vehicle speed and the postedspeed limit.

The imaging system may be also or otherwise operable to control aheadlamp setting or headlamp beam direction of the vehicle in responseto detected headlamps or taillights or other objects of interest alongthe path of travel of the vehicle. The imaging system may detect objectsof interest, such as objects that may pose a threat to the vehicle orlane markers or other objects, and may display information regarding theobjects or other information to the driver of the vehicle, such as at avideo display screen or laser display or heads up display or the like.

Therefore, the present invention provides an imaging system that isoperable to detect and recognize and read traffic control signage alongthe side (and/or above) the road along which the vehicle is traveling.The imaging system may then display information to the driver of thevehicle regarding the detected and recognized signage. The imagingsystem may provide a warning or alert signal to the driver if an unsafeor unwanted driving condition is encountered, such as when the vehicleis traveling at a speed that is substantially over the speed limit or isapproaching a turn at too high a speed or the like.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a side elevation of a portion of a vehicle embodying animaging system in accordance with the present invention;

FIG. 2 is a block diagram of an imaging system in accordance with thepresent invention;

FIG. 3 is a sectional view of an interior rearview mirror assemblyhaving a display device in accordance with the present invention;

FIG. 4 is a sectional view of another interior rearview mirror assemblyhaving another display device in accordance with the present invention;

FIG. 5 is a sectional view of an interior rearview mirror assemblyincorporating a laser display device in accordance with the presentinvention; and

FIG. 6 is a sectional view of an accessory module having an imagingdevice in accordance with the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings and the illustrative embodiments depictedtherein, a vehicle 10 includes an imaging system 12, which includes animaging sensor or image capture device or camera 14, which capturesimages of a scene occurring exteriorly of the vehicle 10 (FIGS. 1 and2). Imaging system 12 includes a control including an image processor16, which receives data from imaging device 14. The image processor 16processes the captured images or captured image data and may identifytraffic control signage (such as stop signs, speed limit signs, exitsigns and the like) and may identify the characters, numbers and/orinformation that is printed or formed or established on the faces of thesigns and may generate an output signal in response to the identifiedcharacters/numbers/information. The imaging system 12 includes a displayor display device 18, which may be positioned within the vehicle (suchas at an interior rearview mirror assembly of the vehicle or at anaccessory module (such as located at an upper portion of the windshield)of the vehicle or at an instrument panel of the vehicle or at anoverhead console of the vehicle or the like) and which displaysinformation in response to image processor 16 processing the capturedimages, as discussed below.

The imaging device 14 may comprise an imaging array sensor, such as aCMOS sensor or a CCD sensor or the like, such as described in U.S. Pat.Nos. 5,550,677; 5,670,935; 5,796,094; 6,498,620; 5,877,897; 6,396,397and 6,313,454, and U.S. patent application Ser. No. 10/421,281, filedApr. 23, 2003, now U.S. Pat. No. 7,004,606, which are herebyincorporated herein by reference. In a preferred embodiment, the imagingsystem 12 may include a lens element or optic between the imaging device14 and the exterior scene. The optic may comprise an asymmetric optic,which focuses a generally central portion of the scene onto the imagingdevice, while providing classical distortion on the periphery of thescene or field of view.

In the illustrated embodiment, the imaging device 14 is mounted at or inan accessory module or pod 20 and is arranged to have a field of viewforward of the vehicle. The imaging device 14 thus may capture images ofa forward scene as the vehicle is traveling forwardly along a road orstreet or highway or the like. Optionally, the imaging device may bepositioned elsewhere, such as at or in the interior rearview mirrorassembly 22, or at or in an accessory module or windshield electronicsmodule or the like (as discussed below), without affecting the scope ofthe present invention.

Display 18 of imaging system 12 may be positioned in the vehicle and maybe readily viewable by the driver of the vehicle. For example, display18 may be positioned in the interior rearview mirror assembly 22 and maybe viewable at the reflective element of the mirror assembly or at oraround the reflective element or bezel portion, such as at the chin oreyebrow region of the mirror assembly. Optionally, the display device 18may be at or in or associated with an accessory module or windshieldelectronics module or the like at or near the interior rearview mirrorassembly, such as an accessory module or windshield electronics moduleof the types described in U.S. patent application Ser. No. 10/355,454,filed Jan. 31, 2003, now U.S. Pat. No. 6,824,281; and Ser. No.10/456,599, filed Jun. 6, 2003, now U.S. Pat. No. 7,004,593, and/or U.S.Pat. Nos. 6,690,268; 6,250,148; 6,341,523; 6,593,565 and 6,326,613,and/or in PCT Application No. PCT/US03/40611, filed Dec. 19, 2003, whichare hereby incorporated herein by reference. Optionally, the displaydevice may be at or in an overhead console (such as a console of thetypes described in PCT Application No. PCT/US03/40611, filed Dec. 19,2003, which is hereby incorporated herein by reference) or elsewhere inthe vehicle, such as in the instrument panel of the vehicle or the like,without affecting the scope of the present invention.

Display or display device 18 may comprise any type of display element ordevice or screen, without affecting the scope of the present invention.For example, display device 18 may comprise a backlit display, which maybe laser-etched or otherwise formed on or placed on (such as via anappliqué or the like) the surface of the reflective element of themirror assembly (such as via removing the reflective coating of thereflective element to form a desired port or icon or character and/orsuch as by utilizing aspects described in U.S. Pat. No. 4,882,565,issued to Gallmeyer, which is hereby incorporated herein by reference)to allow light from corresponding illumination sources or elements topass through the reflective element to illuminate the appropriate portor icon or character for viewing by the driver or occupant of thevehicle, such as described in U.S. patent application Ser. No.10/456,599, filed Jun. 6, 2003, now U.S. Pat. No. 7,004,593; and/or Ser.No. 11/029,695, filed Jan. 5, 2005, now U.S. Pat. No. 7,253,723, whichare hereby incorporated herein by reference. Optionally, the displaydevice may comprise a video screen (such as the types described in PCTApplication No. PCT/US03/40611, filed Dec. 19, 2003, and/or U.S.provisional application Ser. No. 60/630,061, filed Nov. 22, 2004, whichare hereby incorporated herein by reference), or may comprise a displayon demand/transflective type of display or the like at the reflectiveelement of the mirror assembly (where the presence of the display deviceor element may be substantially masked or not readily viewable unlesspowered, such as the types described in U.S. Pat. Nos. 6,690,298;5,668,663 and/or 5,724,187, and/or in U.S. patent application Ser. No.10/054,633, filed Jan. 22, 2002, now U.S. Pat. No. 7,195,381; and/orSer. No. 10/993,302, filed Nov. 19, 2004, now U.S. Pat. No. 7,338,177,which are all hereby incorporated herein by reference), or may comprisea heads up display that projects the display information for viewing infront of the driver of the vehicle, or may comprise any other type ofdisplay (such as the types described in U.S. Pat. Nos. 5,530,240 and/or6,329,925, which are hereby incorporated herein by reference) or thelike, without affecting the scope of the present invention. The displaydevice may include one or more display elements, such as illuminationsources, such as vacuum fluorescent (VF) elements, liquid crystaldisplays (LCDs), light emitting diodes (LEDs), such as inorganic LEDs ororganic light emitting diodes (OLEDs), electroluminescent (EL) elementsor the like, without affecting the scope of the present invention.

Optionally, the display may comprise a video display screen that isselectively viewable, such as at or near the mirror assembly (such as aslide out display of the types described in PCT Application No.PCT/US03/40611, filed Dec. 19, 2003; and/or U.S. provisional applicationSer. No. 60/630,061, filed Nov. 22, 2004, which are hereby incorporatedherein by reference) or through the reflective element of the mirrorassembly. For example, and with reference to FIG. 3, a mirror assembly122 includes a reflective element 130 and a display device or element118 positioned behind the reflective element 130 and within the mirrorhousing or casing 132. The reflective element 130 may comprise a fourthsurface electro-optic reflective element assembly, such as a fourthsurface electrochromic reflective element assembly, and has a reflectivecoating or paint layer 134 on the rear or fourth surface 130 a of thereflective element assembly. Examples of such fourth surface reflectiveelement assemblies are described in U.S. Pat. Nos. 6,690,268; 5,140,455;5,151,816; 6,178,034; 6,154,306; 6,002,544; 5,567,360; 5,525,264;5,610,756; 5,406,414; 5,253,109; 5,076,673; 5,073,012; 5,117,346;5,724,187; 5,668,663; 5,910,854; 5,142,407 and/or 4,712,879, which arehereby incorporated herein by reference. Optionally, the reflectiveelement may comprise a prismatic reflective element with a reflectivecoating or paint layer on its rear surface. As shown in FIG. 3, thereflective coating 134 is removed or otherwise not present (such as bymasking the reflective element surface during the coating process) at adisplay region 136. The mirror assembly 122 includes a movable reflector138, such as a small mirror or reflective element or the like, that isselectively positionable at the display region 136 to provide areflectant element at the display region, so that substantially theentire reflective element 130 reflects to the driver or occupant of thevehicle when the movable reflector 138 is positioned at the displayregion.

When it is desired to display information to the driver or occupant ofthe vehicle, movable reflector 138 may be selectively moved, such as bymoving the reflector rearward of the rear surface of the reflectiveelement 130 and then to a side of (or above or below) the display region136. The display element 118 is positioned generally rearward of thedisplay region 136 so that the display element 118 may be viewablethrough the display window or region when the movable reflector isremoved from the display window or region. Optionally, the displayelement 118 may move toward the display region and may engage or contactthe display region of the reflective element 130 to enhance viewing ofthe display information through the reflective element. When the displayinformation is no longer desired or needed, the display element may moverearward of the reflective element and the movable reflector may moveback to the initial position at the display region.

The display element may be operable to display information relating to arear vision system of the vehicle, a navigation and/or compass system ofthe vehicle, a telematics system of the vehicle or any other vehiclesystem. The movable reflector may be selectively moved and the displayelement may be selectively activated and/or moved in response to a userinput (such as a voice command or manual input at a button or switch orsensor or the like), or may be selectively activated and/or movedautomatically, such as in response to a triggering event, such as whenthe vehicle is shifted into reverse for a backup system or when thevehicle approaches a waypoint for a navigational system (such as anavigational system of the types described in PCT Application No.PCT/US03/40611, filed Dec. 19, 2003; and/or U.S. provisional applicationSer. No. 60/611,796, filed Sep. 21, 2004, which are hereby incorporatedherein by reference), or other triggering events or the like.

The display element and/or movable reflector may be moved via actuationof an electromagnetic drive motor to move the element/reflector to theappropriate location. Optionally, and particularly for applicationswhere the mirror assembly includes compass circuitry for a compasssystem of the vehicle, the mirror assembly or system may include acontrol that may inhibit data sampling by the compass circuitry when thedisplay element and/or movable reflector are moving. For example, thecontrol or system may limit or inhibit data sampling by the compasscircuitry when the display is activated or deactivated and/the movablereflector is moving (or when a slide out display is moving (such as aslide out display of the types described in PCT Application No.PCT/US03/40611, filed Dec. 19, 2003, and/or U.S. provisional applicationSer. No. 60/630,061, filed Nov. 22, 2004, which are hereby incorporatedherein by reference). The control or system thus may interact with thecompass or compass circuitry or control to reduce or limit orsubstantially preclude magnetic interference of the compass systemduring operation of the drive motor (such as an electromagnetic motor)of the movable reflector or slide out display or the like, in order tolimit or substantially preclude the capturing of corrupting data thatmay occur during operation of the electromagnetic motor of the displayor movable reflector. The control or system may be operable to limit orinhibit operation of or data collection by the compass circuitry orsystem during operation of other electromagnetic components of themirror assembly or accessory module or the like.

Optionally, the display may comprise a steerable laser displaypositioned within an accessory module or with the interior rearviewmirror assembly. For example, and with reference to FIG. 4, an interiorrearview mirror assembly 222 includes a steerable laser display device218, such as a controllable or programmable display device that isoperable to project illumination in a controlled or programmed manner.The laser display device 218 may project illumination in a scanningmovement, such as at about 30 frames per second, to project an image asthe laser scans through multiple rows and/or columns at an image viewingarea or display region. In the illustrated embodiment, the scanningdisplay device 218 projects illumination onto an angled reflector 238,which reflects or directs the illumination through a display region 236of the reflective element 230. The reflective element 230 may comprise atransflective electro-optic reflective element assembly, such that thedisplay information may provide a display on demand display (such as thetypes disclosed in U.S. Pat. Nos. 6,690,268; 5,668,663 and/or 5,724,187,and/or in U.S. patent application Ser. No. 10/054,633, filed Jan. 22,2002, now U.S. Pat. No. 7,195,381; PCT Application No. PCT/US03/29776,filed Sep. 9, 2003; and/or PCT Application No. PCT/US03/40611, filedDec. 19, 2003, which are all hereby incorporated herein by reference)that is projectable through the reflective element and viewable by thedriver or occupant of the vehicle when the display element is activated,while the reflective element 230 provides sufficient reflectance in thedisplay region when the display element is deactivated.

The laser scanning display element of the present invention thusprovides a programmable display that may display text or graphics orindicia or the like. The display element provides information to thedriver or occupant of the vehicle as a dynamic display. The displayelement also provides enhanced brightness over some known orconventional display elements and may be implemented at a lower costthan other known or conventional display elements or devices, such asmulti-pixel display screens and the like.

Optionally, a variety of display types or screens can be utilized inconjunction with an interior rearview mirror assembly or windshieldelectronics module/accessory module of the present invention. Forexample, any of the liquid crystal type display or video screens (suchas the types disclosed in PCT Application No. PCT/US03/40611, filed Dec.19, 2003, and/or U.S. provisional application Ser. No. 60/630,061, filedNov. 22, 2004, which are hereby incorporated herein by reference) can beutilized. Also, a microdisplay (such as is available from MicroVisionInc. of Bothell, Wash.), in which a single scanner is used to directmultiple light beams simultaneously into separate zones of an image soas to deliver a bright, high resolution, image over a wide field ofview, can be used. Such a microdisplay may utilize conventional surfaceemitting or other types of light emitting diodes (LEDs) as light sourcesto provide an economical display with sharp resolution and high imagebrightness. For example, multiple red, green and blue LEDs or red, blueand green laser diodes can be used to write several million red, green,and blue spots that integrate to form a single high-fidelity image in amega pixel display image. Such scanning display technologies can utilizea biaxial microelectromechanical scanner (MEMS) and otherdisplay/mechanical and electronic devices, such as are disclosed in U.S.Pat. Nos. 6,714,331; 6,795,221 and 6,762,867, which are herebyincorporated herein by reference, and can provide increased spatialresolution. Such displays can deliver an image with a full 30-degreehorizontal field of view or more. Such a microdisplay/MEMS device can,for example, be placed in the mirror housing behind the mirrorreflective element in an interior (or exterior) mirror assembly suchthat the image is projected onto the rear of the mirror reflectiveelement, such as is disclosed in U.S. patent application Ser. No.10/225,851, filed Aug. 22, 2002, now U.S. Pat. No. 6,847,487, which ishereby incorporated herein by reference.

If the mirror reflector of the mirror element is of the transflective(substantially reflective and at least partially transmitting to light)type, the driver or other occupant in the interior cabin of the vehiclecan view the image (being back-projected onto the rear of the mirrorreflective element) by viewing the mirror reflective element. This isbecause the front surface of the reflective element will typicallyreflect about 4 percent of the light incident on the reflective elementtoward the driver of the vehicle. Thus, if the display illumination(projected through the reflective element from behind the reflectiveelement and within the mirror casing) does not dominate or distinguishover the reflectance off of the front surface of the mirror reflectiveelement, the display illumination and information may appear washed outdue to the reflected image that is reflecting off of the front surfaceof the reflective element. Such washout may be particularly noticeableduring high ambient lighting or daytime lighting conditions. Becauseback-projected microdisplays can have a very high image brightness (dueto use of very high brightness LEDs or lasers as illuminators), imagewash-out during driving under high ambient lighting conditions (such ason a sunny day) is reduced using such scanning image microdisplaytechnology compared to use, for example, of TFT LCD displays.

Also, such MEMS technology can be used in a heads-up-display (HUD)system, such as the MicroHUD™ head-up display system available fromMicroVision Inc. of Bothell, Wash. (and such as described in U.S. patentapplication Ser. No. 11/029,695, filed Jan. 5, 2005, now U.S. Pat. No.7,253,723, which is hereby incorporated herein by reference). Thisprovides a compact heads-up display capable of meeting specific size andperformance specifications. For example, MicroVision's MicroHUD™combines a MEMS-based micro display with an optical package of lensesand mirrors to achieve a compact high-performance HUD module thatreflects a virtual image off the windscreen that appears to the driverto be close to the front of the car. This laser-scanning display canoutperform many miniature flat panel LCD display screens because it canbe clearly viewed in the brightest conditions and also dimmed to thevery low brightness levels required for safe night-time driving.

The high-resolution MicroHUD™ display may be completely reconfigurable,enabling virtually any content to be displayed, including video oranimated icons and graphics. Advantageously, such a MicroHUD™ displayunit may be included at or within an interior rearview mirror assemblyor a windshield electronics module/accessory module so as to project itsimage therefrom onto the inner surface of the windshield. This uniquepackaging of a HUD or projection image displayer into an interiorrearview mirror assembly or a windshield electronics module/accessorymodule has advantages over conventional placement of such HUD projectorsinto the dashboard of the vehicle. These advantages include that the HUDimage projector need not find space in an already crowded dashboard(where, for example, a center information cluster may want space orwhere HVAC ducts/components may run). Also, incorporation of the HUDprojector in the likes of the mounting portion of the interior mirrorassembly or into a windshield electronics module/accessory module canallow a HUD display to be provided more readily as an optional accessoryfor the vehicle or as a dealership option or aftermarket device. Avariety of images (such as, for example, iconistic or graphical or videoor textural or alphanumerical or numerical or the like) can bedisplayed, such as information from a side object/blind spot monitoringsystem, such as the types described in U.S. Pat. No. 5,929,786, and/orU.S. patent application Ser. No. 10/427,051, filed Apr. 30, 2003, nowU.S. Pat. No. 7,038,577; and/or Ser. No. 10/209,173, filed Jul. 31,2002, now U.S. Pat. No. 6,882,287, and/or U.S. provisional applicationSer. No. 60/638,687, filed Dec. 23, 2004, which are hereby incorporatedherein by reference.

Also, a full video image captured by the likes of a reversing camera ora forward facing night vision camera or a sidelane-monitoring camera canbe displayed on/via the vehicle windshield (or elsewhere) by the likesof a MicroHUD™ device and, conceptually, thus replacing the exteriormirrors with cameras. For example, a driver sidelane video image and apassenger sidelane video image, both preferably with graphic overlaysthereon, can be displayed at respective sides of the vehicle windshieldvia a MEMS-based display system (such as via a MicroHUD™ HUD displaydevice) and with the image visible to the driver by viewing the vehiclewindshield (such as via an optical image combiner created on the innerglass surface of the windshield and/or onto the polymeric laminatinginterlayer (typically a sheet of polyvinyl butyral or of silicone or thelike) utilized in the laminate windshield).

Optionally, a laser emitter or laser diode or the like may be positionedwithin the mirror casing and behind the reflective element, and may beused to emit radiation onto a reflector (such as amicroelectromechanical scanner (MEMS)) within the mirror casing thatreflects the radiation toward and through the mirror reflective elementfor viewing by a driver of the vehicle (such as by utilizing aspectsdescribed in U.S. patent application Ser. No. 10/225,851, filed Aug. 22,2002, now U.S. Pat. No. 6,847,487; and/or U.S. provisional applications,Ser. No. 60/607,963, filed Sep. 8, 2004; Ser. No. 60/642,227, filed Jan.7, 2005; and Ser. No. 60/644,903, filed Jan. 19, 2005, which are allhereby incorporated herein by reference).

Such a laser scanning display device may provide enhanced displaycharacteristics for enhanced viewing of the display at the reflectiveelement by the driver of the vehicle. Typically, in order to use a laserto back light a display area (such as an area of about two cm square orthereabouts), the laser beam may be projected through an optic thatbroadens the beam to the desired size, whereby the intensity of the beamis reduced. An advantage of such scanning display technologies is theintensity of the display delivered, and thus its ability to be seen evenunder high ambient driving conditions (such as a sunny day). Forexample, should a standard backlit TFT LCD display be placed behind atransflective mirror element in the likes of an interior rearview mirrorassembly, the front or first surface reflection off the front glasssurface (typically around 4 percent of the light incident thereon) oftenfar exceeds the intensity of the light transmitted through thetransflective mirror reflective element used. Such transflective mirrorsalso reflect coincident with the reflection off the front surface, andthus further exasperate the washout of the display image beingtransmitted/emitted through the reflective element. Even if thereflective coating is locally fully removed to create a lighttransmitting window, reflectivity off the front glass surface oftencauses display washout and inability to appropriately read what is beingviewed at the display. This is particularly problematic for videodisplay (such as for the likes of a rear backup event or side lanemaneuver event or the like).

Thus, one advantage of use of such a scanning display technology (suchas described in further detail below) is that the full intensity of thelaser is used, but by using the movable mirror/reflector of themicroelectromechanical scanner (MEMS), the narrow point-like, super highintensity beam rapidly moves across the display image dimension at arate that is faster than the eye/brain can register, such that theeye/brain perceives a continuous (or substantially continuous) superbright image. Thus, using the concepts of the present invention asdescribed below, a full video image can effectively be projected throughor on a surface of the rearview mirror reflective element in a mannernot unlike what can be seen during outdoor laser displays or the like(such as when images and video is laser written on the sides ofbuildings or clouds or the like). Also, multiple lasers of the samecolor can be focused so that their beams coincide at roughly the samepoint on the MEMS reflector so that the intensity of any one imageelement as written is correspondingly enhanced.

For example, and with reference to FIG. 5, an interior rearview mirrorassembly 310 may be pivotally or adjustably mounted to an interiorportion of a vehicle, such as via a double ball mounting or bracketassembly 312. For example, the bracket assembly 312 may include a mirrormount 312 a that is mountable to a mounting button 313 adhered or bondedto an interior surface 311 a of a vehicle windshield 311. The bracketassembly 312 may also include a mounting arm 312 b that is pivotallyattached to the mirror mount 312 a at a first pivot joint 312 c and thatis pivotally attached to the mirror casing or mirror head at a secondpivot joint 312 d. Other means for adjustably mounting the mirror headto an interior portion of the vehicle may be implemented withoutaffecting the scope of the present invention.

Mirror assembly 310 includes an electro-optic or electrochromicreflective element 314 supported at or in a housing or casing 318. Themirror assembly 310 includes a scanning display device 326 that isoperable to display information (such as text, alphanumeric characters,icons, images, video images, or other indicia or information or thelike) at the reflective element 314 for viewing by a driver of thevehicle. Advantageously, display device 326 is housed behind (to therear of) the mirror reflective element and thus is within mirror casing318. Thus, the automaker may acquire and install mirror assembly 310(with the scanning display capability included) across a variety ofvehicle models and lines. Reflective element 314 includes a frontsubstrate 320 and a rear substrate 322 and an electro-optic medium 324disposed therebetween with a seal 325 encompassing the electro-opticmedium, as is known in the electro-optic mirror arts. The frontsubstrate 320 includes a transparent conductive coating or layer 321 atits rear surface (commonly referred to as the second surface of thereflective element), while the rear substrate 322 includes a conductivecoating 323 at its front or forward surface (commonly referred to as thethird surface of the reflective element).

The reflective element may comprise a transflective reflective elementthat allows light from the display device 326 to pass therethrough forviewing by the driver of the vehicle, such as by utilizing principlesdescribed in U.S. Pat. Nos. 6,690,268; 5,668,663 and/or 5,724,187,and/or in U.S. patent application Ser. No. 10/054,633, filed Jan. 22,2002, now U.S. Pat. No. 7,195,381; and/or Ser. No. 11/021,065, filedDec. 23, 2004, now U.S. Pat. No. 7,255,451; and/or PCT Application No.PCT/US03/29776, filed Sep. 9, 2003; and/or PCT Application No.PCT/US03/35381, filed Nov. 5, 2003; and/or U.S. provisionalapplications, Ser. No. 60/630,061, filed Nov. 22, 2004; Ser. No.60/629,926, filed Nov. 22, 2004; Ser. No. 60/531,838, filed Dec. 23,2003; Ser. No. 60/553,842, filed Mar. 17, 2004; Ser. No. 60/563,342,filed Apr. 19, 2004; Ser. No. 60/644,903, filed Jan. 19, 2005; Ser. No.60/667,049, filed Mar. 31, 2005; Ser. No. 60/653,787, filed Feb. 17,2005; Ser. No. 60/642,227, filed Jan. 7, 2005; Ser. No. 60/638,250,filed Dec. 21, 2004; Ser. No. 60/624,091, filed Nov. 1, 2004; and Ser.No. 60/609,642, filed Sep. 14, 2004, and/or PCT Application No.PCT/US03/40611, filed Dec. 19, 2003, which are all hereby incorporatedherein by reference. Optionally, use of an elemental semiconductormirror, such as a silicon metal mirror, such as disclosed in U.S. Pat.Nos. 6,286,965; 6,196,688; 5,535,056; 5,751,489 and 6,065,840, and/or inU.S. patent application Ser. No. 10/993,302, filed Nov. 19, 2004, nowU.S. Pat. No. 7,338,177, which are all hereby incorporated herein byreference, can be advantageous because such elemental semiconductormirrors (such as can be formed by depositing a thin film of silicon) canbe greater than 50% reflecting in the photopic (SAE J964a measured),while being also substantially transmitting of light (up to 20% or evenmore). Such silicon mirrors also have the advantage of being able to bedeposited onto a flat glass substrate and to be bent into a curved (suchas a convex or aspheric) curvature, which is also advantageous sincemany passenger-side mirrors are bent or curved.

Display device 326 comprises a scanning beam display system thatincludes a plurality of laser light sources or diodes 328 a, 328 b, 328c, a controller 330 and a microelectromechanical scanner (MEMS) 332. Thedisplay device 326 is contained within the interior casing 318 of mirrorassembly 310. The controller 330 receives and/or generates image signalsthat control the intensity, mix and on-time of the light output by thethree laser diodes 328 a, 328 b, 328 c. The controller 330 alsoestablishes the coordinates for the movable elements of the MEMSassembly 332 so that the individual picture elements (pixels) of thedisplayed image (as displayed at the display area or region 333 at thereflective element 314) are created for view by the driver or othervehicular occupant. For monochrome (one-color) systems, only one laserdiode source may be used. Optionally, for full-color displays, threelight sources (e.g., red, green and blue) are modulated and merged toproduce an image element of the appropriate color. Under the control ofcontroller 330, a horizontal and vertical scanner or a singlemicroelectromechanical scanner (MEMS) 332 directs the light beamsreceived from laser diodes 328 a, 328 b, 328 c, and projects them ontothe rear of (and/or into the body of) mirror reflective element 314 tocreate the image viewed. Optics (not shown) may be included as desiredto achieve the desired spatial and resolution dimensions displayed.

For example, mirrors and/or lens elements or other refractive ordiffractive and/or reflective optical elements can be used to projectthe rapidly scanned beam or beams of light onto the rear of the mirrorelement (and/or into the body thereof) to create the image seen. Such ascanned-beam automotive mirror display can deliver very high resolution,very high intensity images, with the resolution being limitedprincipally by diffraction and optical aberrations in the light sourcesused within the mirror casing. Optionally, the rear surface 322 a of therear substrate 322 of the reflective element 314 may include a diffusercoating or layer/combiner 334 or other diffuser means or the like, andthe diffuser coating or layer or area may be over substantially theentire rear surface 322 a or may be over only that portion of the rearor fourth surface rastered by light reflected off the MEMS 332 thatcreates the display image. Also, and optionally, diffuser coatingsand/or layers/combiners may be included within the body of the mirrorreflective element, such as on the third surface of the electro-opticreflective element.

Although illustrated as a transflective mirror element, the reflectivecoating may be locally removed from a non-transflective mirror elementto create a window for viewing the display thereat or therethrough. Thewindow region may include a diffuse coating and/or layer/combiner or thelike, such as on the rear surface of the reflective element (such as ifthe reflective element is an electro-optic or electrochromic reflectiveelement or a non-electro-optic or prismatic reflective element) or onthe third surface (such as if the reflective element is an electro-opticor electrochromic reflective element), if desired.

The laser diodes may be rastered or scanned at a desired rate over theMEMS reflector so that a generally continuous image is created byreflection off the MEMS and onto/into and as viewed through thereflective element. In the illustrated embodiment, the laser diodes arepositioned to project or emit or radiate their laser beams so that theyare incident on the electromechanically moved portion of the MEMS andwhereby the laser beams are reflected toward the reflective element bythe MEMS reflector.

The MEMS 332 may be positioned within the casing and angled or orientedto reflect illumination or radiation from the laser diodes 328 a, 328 b,328 c toward the rear surface of the reflective element 314. Thereflective surface of the MEMS 332 may be created on a chip, and may beadjusted to provide the desired projection or reflection angle throughthe reflective element 314 for viewing by a driver of the vehicle. TheMEMS reflector may be electrically adjusted and/or electromechanicallyadjusted to provide the appropriate or desired information or icon orimage for the laser beams to project onto and through the reflectiveelement. The laser diodes 328 a, 328 b, 328 c may comprise any laserdiodes, such as, for example, laser diodes of the types commerciallyavailable from Cree Research Inc. of Durham, N.C., which offersdifferent color laser diodes, such as visible red laser diodes and/orblue laser diodes, such as gallium nitride based blue lasers, and othercolors as may be desired, such as, for example, green.

Because of the high intensity illumination provided by such laserdiodes, the intensity at the display region of the reflective elementwill be sufficient to dominate the reflection of the rearward scene offof the front surface of the front substrate of the reflective element,and thus will not appear washed out, even during high ambient lightingconditions, such as on a sunny day or the like. Optionally, theintensity of the laser diodes may be adjusted, such as via manualadjustment and/or via automatic adjustment, such as in response to theambient light levels in the cabin of the vehicle or in the vicinity ofthe display. The display information may be associated with anyaccessory or component or feature of the interior rearview mirrorassembly or of the vehicle, such as point-to-point navigationalinstructions, status information for various functions, such aspassenger side airbag status, tire pressure status and/or the like, orcompass heading or temperature information or other information or thelike.

Also, a video display and/or other information display may be located atthe interior mirror assembly (or at a windshield electronicsmodule/accessory module) that utilizes aMicro-Electro-Mechanical-Systems (MEMS) structure combined with thinfilm optics, such as is available Iridigm of San Francisco, Calif. underthe tradename iMoD™ technology. This display technology (such as isdescribed in U.S. Pat. Nos. 6,794,119; 6,741,377; 6,710,908; 6,680,792;6,674,562; 6,650,455; 6,589,625; 6,574,033; 5,986,796 and 5,835,255,which are hereby incorporated herein by reference) is designed todeliver lower power consumption and excellent display image quality, andcan withstand extreme temperatures and can be viewed in substantiallyany environment, including bright sunlight.

Although shown and described as being incorporated into an electro-opticor electrochromic interior rearview mirror assembly, it is envisionedthat the scanning beam display system may be incorporated into aprismatic interior rearview mirror assembly or a transflective prismaticrearview mirror assembly (such as by utilizing principles described inPCT Application No. PCT/US03/29776, filed Sep. 19, 2003; U.S. patentapplication Ser. No. 11/021,065, filed Dec. 23, 2004, now U.S. Pat. No.7,255,451; and/or Ser. No. 10/993,302, filed Nov. 19, 2004, now U.S.Pat. No. 7,338,177, which are all hereby incorporated herein byreference). Optionally, the laser scanning beam display system may beincorporated into an exterior rearview mirror assembly without affectingthe scope of the present invention. For exterior rearview mirrorapplications, the display system may function to display blind spotdetection icons or information, or turn signals or security lights orthe like, at the reflective element of the exterior rearview mirrorassembly of the vehicle. For example, a non-electro-optic/fixedreflectivity reflector may use an elemental semiconductor mirror, suchas a silicon metal mirror, such as disclosed in U.S. Pat. Nos.6,286,965; 6,196,688; 5,535,056; 5,751,489 and 6,065,840, and/or in U.S.patent application Ser. No. 10/993,302, filed Nov. 19, 2004, now U.S.Pat. No. 7,338,177, which are all hereby incorporated herein byreference, can be advantageous because such elemental semiconductormirrors (such as can be formed by depositing a thin film of silicon) canbe greater than 50% reflecting in the photopic (SAE J964a measured),while being also substantially transmitting of light (up to 20% or evenmore). Such silicon mirrors also have the advantage of being able to bedeposited onto a flat glass substrate and to be bent into a curved (suchas a convex or aspheric) curvature, which is also advantageous sincemany passenger-side mirrors are bent or curved.

Optionally, the display may comprise a laser emitter or laser diode orthe like, which may be positioned within the mirror casing and behindthe reflective element, and may be used to emit radiation onto areflector (such as a microelectromechanical scanner (MEMS)) within themirror casing that reflects the radiation toward and through the mirrorreflective element for viewing by a driver of the vehicle (such as byutilizing aspects described in U.S. patent application Ser. No.10/225,851, filed Aug. 22, 2002, now U.S. Pat. No. 6,847,487; and/orU.S. provisional applications, Ser. No. 60/644,903, filed Jan. 19, 2005;Ser. No. 60/642,227, filed Jan. 7, 2005; and/or Ser. No. 60/607,963,filed Sep. 8, 2004, which are hereby incorporated herein by reference).

The light emitting device, such as a laser diode or light emitting diode(LED) or the like (such as described in U.S. provisional applications,Ser. No. 60/644,903, filed Jan. 19, 2005; Ser. No. 60/642,227, filedJan. 7, 2005; and/or Ser. No. 60/607,963, filed Sep. 8, 2004, which arehereby incorporated herein by reference), of the display may becontrolled by a controller, which may modulate the intensity or on/offcharacteristic of the emitted light while the light emitting device orlaser is rastered (or moved rapidly back and forth in generallyhorizontal or vertical scanning lines over a display area), in order tocreate the desired display via the points where the light emittingdevice is intensified or activated. Because the laser diode may berastered at a high rate over substantially the entire display area butonly activated/intensified at appropriate locations to form the desireddisplay, the narrow point like, super high intensity beam (that isactivated/intensified/modulated as the laser diode is rapidly movedacross the display image dimension at a rate that is faster than theeye/brain can register) is perceived by the human eye/brain as acontinuous (or substantially continuous) super bright image, even thoughonly one light “dot” or beam may actually be present at a time at thedisplay. A person viewing the display thus would see the display as thedesired form or character and substantially constantly and brightlyilluminated by the rastered and modulated laser diode.

Optionally, the light emitting device may be substantially constantlyactivated and directed/rastered toward a display window, such as aliquid crystal display (LCD) or the like, with a window established inthe desired form, so that light emitted by the light emitting device(such as a laser diode, a light emitting diode (LED) or an organic lightemitting diode (OLED) or the like) projects or shines through thedisplay window/element, such that the display character or icon orinformation or video or the like is viewable at the reflective elementby the driver of the vehicle. The display window may comprise asubstantially transparent or translucent shape or character or icon orthe like, with a darkened or substantially opaque area surrounding thewindow, such that light emitted by the light emitting device passesthrough or transmits through the window, but is substantially blocked orattenuated by the surrounding opaque area of the display. The LCDdisplay may be operable to adjust the window and opaque regions toadjust or modulate or change or control the information being displayedby the light passing through the display. For applications where thelight emitting device may be rastered at a high rate over substantiallythe entire display area (such as over the LCD), and with theillumination beam (such as the narrow point like, super high intensitybeam of a laser emitting device) rapidly moving across the display imagedimension at a rate that is faster than the eye/brain can register, theeye/brain perceives a continuous (or substantially continuous) brightimage, even though only one light “dot” or beam may be present at a timethrough the display window. The light emitting device thus may beconstantly or substantially constantly activated/energized, with thedisplay being formed/created by the window through which the lightpasses as the light beam is rastered or scanned over the display device.A person viewing the display thus would see the display as the characterof the display window as substantially constantly and brightlyilluminated by the rastered laser diode or other light emitting device,such as an LED or OLED or the like.

Note that is desirable, and in many cases preferable, that the laserlight source be only enabled when the MEMS unit is functioning andcausing a rastering or the like of the reflected laser beam. This is tolimit or substantially preclude or prevent the possibility of the laserbeam being stationary for any prolonged period with a concomitantpossibility of eye damage to viewer in the vehicle. Thus, thecircuitry/software controlling activation/illumination of the laserlight source can be tied to the circuitry/software controllingactivation/movement of the movable reflector of the MEMS unit, such thatshould the system fail and the MEMS unit not move or cease rastering,then the laser source is extinguished/turned off so that danger to aviewer from laser eye burn or the like is obviated.

Optionally, a projected information display and/or virtual human machineinterface (HMI) may be created at a surface of an interior mirrorassembly or a windshield electronics module/accessory module utilizing avirtual data entry device system, such as is disclosed in U.S. Pat. Pub.No. 20020075240, published Jun. 20, 2002, which is hereby incorporatedherein by reference. Thus, an optically generated image of a key-entrytablet or an input menu or user-actuation button/input or an icon or aninformational message or the like can be projected, for example, onto asurface of the interior rearview mirror or elsewhere within the cabin ofthe vehicle. The projected image may include at least one inputzone/user interface zone that is actuatable by an action performedthereon or thereat by a user. The system includes a sensor operative tosense the action performed on or at the at least one input zone, and togenerate signals in response to the detected action. A control orprocessor in communication with the sensor is operable to process thesignals for performing an operation associated with the at least oneinput zone.

For example, a template of the desired interface (such as a keyboard orinput options or the like) may be projected onto an interface surface(such as the reflective element of the interior mirror assembly). Thetemplate is produced by illuminating an optical element (such as aholographic optical element) with a laser diode (such as a red laserdiode or the like). Because the template functions only as a referencefor the user and is not involved in the detection process, the templatemay optionally be printed at a desired surface, such as at a portion ofthe reflective element or casing of the mirror assembly (or at a casingor element of a windshield electronics module or accessory module or thelike).

An infrared plane of light may be generated at and slightly spaced fromand parallel to the interface surface. The light may be substantiallyinvisible to the user and is positioned just a few millimeters away fromthe interface surface (such as along the first surface of the reflectiveelement and a few millimeters toward the driver or toward the rear ofthe vehicle from the first surface of the reflective element).Accordingly, when a user touches a portion of the projected interface atthe interface surface (for example, the first surface of the reflectiveelement of the interior mirror assembly), light is reflected from theplane in the vicinity of the respective input or key that was “touched”and directed toward the sensor module.

The reflected light from the user interaction with the interface surfaceis received by or imaged onto an imaging array sensor, such as a CMOSimage sensor or the like, in a sensor module. The reflected light maypass through an infrared filter before being imaged onto the CMOSsensor. The sensor control or processor or chip then may conduct areal-time determination of the location of the reflected light, and maybe operable to track multiple reflection events substantiallysimultaneously, and can thus support both multiple inputs/keystrokes andoverlapping cursor control inputs and the like. The micro-controller(which may be positioned in the sensor module) receives the positionalinformation corresponding to the light flashes from the sensor controlor processor, and interprets the events and communicates them through anappropriate interface to the appropriate external device or devices.

The projected interface and sensor system thus may provide a keypad orinput interface at the reflective element for actuation/use by thedriver or occupant of the vehicle. The keypad or input interface may beprojected onto or at the reflective element only when it is desired tobe used, such that the reflective element is substantially unaffected bythe incorporation of the interface and sensor system at the interiorrearview mirror assembly. The sensor may detect the input actionperformed/selected by the user and the control may then control oractivate/deactivate or modulate or adjust the appropriate accessory orsystem or device of the vehicle.

The information or input interface that is projected may provide variousinputs/actions, such as, for example, inputs for a video display of thevehicle (such as disclosed in U.S. Pat. Nos. 5,760,962 and/or 5,877,897;and/or PCT Application No. PCT/US03/40611, filed Dec. 19, 2003, and/orU.S. provisional applications, Ser. No. 60/630,061, filed Nov. 22, 2004;Ser. No. 60/628,709, filed Nov. 17, 2004; Ser. No. 60/614,644, filedSep. 30, 2004; and/or Ser. No. 60/618,686, filed Oct. 14, 2004, whichare hereby incorporated herein by reference), a communications system ofthe vehicle (such as disclosed in U.S. Pat. Nos. 6,717,524; 6,650,233;6,243,003; 6,278,377 and/or 6,420,975, and/or PCT Application No.PCT/US03/30877, filed Oct. 1, 2003, which are hereby incorporated hereinby reference), a navigational system of the vehicle (such as the typesdescribed in U.S. Pat. No. 6,477,464, and U.S. patent application Ser.No. 10/456,599, filed Jun. 6, 2003, now U.S. Pat. No. 7,004,593; Ser.No. 10/287,178, filed Nov. 4, 2002, now U.S. Pat. No. 6,678,614; Ser.No. 10/645,762, filed Aug. 20, 2003, now U.S. Pat. No. 7,167,796; andSer. No. 10/422,378, filed Apr. 24, 2003, now U.S. Pat. No. 6,946,978;and/or PCT Application No. PCT/US03/40611, filed Dec. 19, 2003, whichare hereby incorporated herein by reference), light sources (such as mapreading lights or one or more other lights or illumination sources, suchas disclosed in U.S. Pat. Nos. 6,690,268; 5,938,321; 5,813,745;5,820,245; 5,673,994; 5,649,756; 5,178,448; 5,671,996; 4,646,210;4,733,336; 4,807,096; 6,042,253 and/or 5,669,698, and/or U.S. patentapplication Ser. No. 10/054,633, filed Jan. 22, 2002, now U.S. Pat. No.7,195,381, which are hereby incorporated herein by reference) and/or thelike. Optionally, different interfaces may be provided for differentaccessories or devices or functions, whereby the appropriate interfacefor a particular accessory or device or function may be selected by theuser, and the desired particular function of that accessory or devicemay then be selected and activated or deactivated or controlled by“touching” the appropriate location at the surface (such as the firstsurface of the reflective element) upon which the interface keypad orinput is projected.

Other types of displays or display elements or devices and controls forsuch displays or display elements or devices may be implemented with theimaging system of the present invention, without affecting the scope ofthe present invention.

The imaging system of the present invention may be utilized to identifyparticular traffic control signs or signage by their spectral signatureas well as their geometric organization. For example, red octagons maybe identified as stop signs, yellow triangles as caution signs, and thelike, while black characters on a rectangular white background may beidentified as a speed limit sign (in certain zones or regions orcountries). These capabilities are a result of the present inventionproviding a significant reduction in the amount of data to be processedbecause the image forward of the vehicle is captured in a manner whichpreselects data. Preselection of data is accomplished by configuring theimaging device or sensor array, including the optics thereof, toconsider the spatial, as well as the spectral, characteristics of lightsources and objects in the captured images, such as via utilization ofprinciples described in U.S. Pat. No. 5,796,094, which is herebyincorporated herein by reference.

More particularly, image processor 16 receives an input signal generatedby imaging device 14 and interrogates or processes the imaging deviceoutput to detect traffic control signage in the captured image of theforward scene. The image processor 16 may identify what type of sign isin the captured image based on the geometrical shape of the sign, thesize of the sign and the location of the sign relative to the vehicle orroad. For example, the image processor may process the image todetermine the location of the detected object or sign relative to thefield of view of the imaging device or camera and, thus, relative to thevehicle and to the side of the road where such a sign is expected to befound (typically at the side of the vehicle that is opposite to thedriver's side of the vehicle). The imaging processor may determine theshape, size, color and/or location of the detected sign or object viaany suitable sign recognition and sign informationdelineation/discrimination algorithm/software utilized by the imagingsystem. Such software or algorithm may incorporate any suitableprocessing means, such as by utilizing aspects described in U.S. Pat.Nos. 5,550,677; 5,670,935; 5,796,094; 6,498,620; 5,877,897; 6,396,397;6,353,392 and 6,313,454, and/or U.S. patent application Ser. No.10/427,051, filed Apr. 30, 2003, now U.S. Pat. No. 7,038,577, which arehereby incorporated herein by reference. For example, the imageprocessor may process the image via an edge detection algorithm or thelike, such as described in U.S. Pat. Nos. 6,353,392 and 6,313,454,and/or U.S. patent application Ser. No. 10/427,051, filed Apr. 30, 2003,now U.S. Pat. No. 7,038,577, which are hereby incorporated herein byreference.

In a preferred embodiment, the imaging device comprises an imaging arraysensor that is responsive to light and that includes colored filters ora filter array at or over the pixels of the sensor, such that the pixelsare spectrally responsive to different colors of light, such asdescribed in U.S. Pat. Nos. 5,550,677; 5,670,935; 5,796,094; 6,498,620;5,877,897; 6,396,397 and 6,313,454, which are hereby incorporated hereinby reference. The filters or filter array may be selected to provideenhanced recognition of colors within a selected spectral band or bandsof light. The imaging device and the imaging system thus may haveenhanced recognition of certain colors that may be expected to be usedon the signs or signage of interest (or may have enhanced rejection ofcertain spectral bands that may not be used on signage of interest).

Such traffic control signage, such as speed limit signs, exit signs,warning signs, stop signs, yield signs and/or the like, is typicallyregulated and various types of these signs must have certain specified,standard geometric shapes (such as a triangle for a yield sign, anoctagon for a stop sign and the like), and must be at a particularheight and at a particular location at or distance from the side of theroad, and must have a specific type/color of lettering on a specificcolored background (for example, a speed limit sign is typically apredefined shape, such as rectangular or circular, and has alphanumericcharacters or letters and/or numbers that are a contrast color to abackground color, such as black letters/numbers on a white background,while an exit sign typically has a different shape and/or contrastcolors, such as white lettering on a green background). The imagingdevice is arranged at the vehicle, preferably in the interior cabin andviewing through the windshield (and thus protected from the outdoorelements, such as rain, snow, etc.), with a field of view thatencompasses the expected locations of such signage along the side ofroads and highways and the image processor may process the capturedimage to determine if the captured images encompass an object or signthat is at the expected location and that has the expected size, colorand/or shape or the like. Therefore, the imaging processor 16 mayreadily determine what type of sign is detected by its geometric shape,size, color, text/characters and its location relative to the imagingdevice and the vehicle.

Preferably, the image processing algorithm or software includes a signrecognition stage or step or portion or process or routine thatprocesses the image to determine whether the detected sign or object isof interest and, if so, what type of sign is detected. Once the signrecognition stage is satisfied, the image processing algorithm orsoftware proceeds or passes to a character recognition stage or step orportion or process or routine, where the image is processed further todetermine or recognize the characters (such as alphanumeric characters,letters, numbers or icons or indicia or the like) printed or formed orestablished on the face of the sign, in order to determine theinformation conveyed by the characters or icons or indicia on the faceof the sign. The processor involved thus may only be busied with thecharacter recognition stage once the preceding sign recognition stagehas recognized and determined that a speed limit sign (or other sign orsignage of interest) within the field of view. The algorithm processedby the image processor may include false signal and/or error reductionroutines and protection, whereby instances of errant or unintended/falsereadings of items or objects other than genuine signage are reduced orsuppressed.

Once the type of sign is determined, the imaging system may process thecharacters (which may be alphanumeric characters or numbers or lettersor icons or the like) printed or formed or established on the sign, andno further processing of the sign's size or shape or color or the likeneed be conducted. The imaging system thus may process the images onlyenough to determine the type of sign and to determine the characters orinformation on the face of the sign if necessary, such that reducedprocessing may be achieved in certain circumstances where the sign typeis readily identifiable. For example, a stop sign may be readilyidentified by its shape and color, such that no further processing maybe required to determine the sign type or the characters or informationon the face of the sign.

It is further envisioned that the detected color of the charactersand/or background may be compared to the regulation or specified signcolors for daytime and/or nighttime lighting conditions. For example, ifthe vehicle is traveling during high ambient light conditions (which maybe determined by processing the output of the imaging device or via aseparate ambient light sensor or the like), such as during the daytime,the imaging system may determine the type of sign detected in responseto matching the detected sign color to the specified or regulated colorsfor the sign during daytime lighting conditions, while if the vehicle istraveling during low ambient light conditions, such as belowapproximately 200 lux or thereabouts, such as during nighttime, theimaging system may determine the type of sign detected by matching thedetected sign color to the specified or regulated colors for the signunder headlamp or auxiliary lighting conditions such as typically occurat nighttime.

In different countries or regions, and even along different types ofroads or highways, the signage regulations may be different, and theimaging processor may be adjusted accordingly to adapt to the differentregulations. It is further envisioned that the imaging system may beautomatically adjusted or adapted to the sign regulations in effect atthe current location of the vehicle. The current location of the vehiclemay be determined via a vehicular navigational system or globalpositioning system (GPS) or the like, such as the types described inU.S. Pat. Nos. 6,477,464; 5,924,212; 4,862,594; 4,937,945; 5,131,154;5,255,442 and/or 5,632,092, and/or U.S. patent application Ser. No.10/456,599, filed Jun. 6, 2003, now U.S. Pat. No. 7,004,593; Ser. No.10/287,178, filed Nov. 4, 2002, now U.S. Pat. No. 6,678,614; Ser. No.10/645,762, filed Aug. 20, 2003, now U.S. Pat. No. 7,167,796; and Ser.No. 10/422,378, filed Apr. 24, 2003, now U.S. Pat. No. 6,946,978; and/orPCT Application No. PCT/US03/40611, filed Dec. 19, 2003, which are allhereby incorporated herein by reference.

Optionally, a user input may be provided to selectively input thelocation or zone or region of the vehicle to establish the appropriatesetting for the imaging system. For example, a user may change from an“imperial” setting (such as used in the U.S.), where the numbers may beinterpreted by the imaging system as being in miles per hour, to a“metric” setting, where the numbers may be interpreted by the imagingsystem as being in kilometers per hour, such as when the driver drivesthe vehicle from the U.S. to Canada. Optionally, if the vehicle has aglobal positioning system (GPS), the setting for a particular locationor zone at which the vehicle is located may be automatically set to theappropriate units or setting in response to a signal from the globalpositioning system that is indicative of the current location orposition of the vehicle. Other zones or regions may be selectively ormanually input or automatically set to set or calibrate the imagingsystem for the particular zone or region or country in which the vehicleis traveling (where the numbers may be interpreted according to theunits used in that zone or region or country and where the detectedsigns or objects are compared to the expected sign shapes, sizes, colorsand the like of that zone or region or country).

Optionally, the expected or recognized or accepted sign size, shape,color, etc. may be looked up in a table or database or the like by theimage processor, in order to determine if a detected sign or object iswithin the expected or specified parameters of a particular type of signand, thus, to determine if the detected object or sign qualifies as aparticular type of traffic control sign. For example, if the imagingsystem detects an object that is generally in the area and of the sizeof a traffic control sign, the system may further analyze the signparameters in view of a table or listing or database of parameters ofvarious signs along different types of roads or the like, in order todetermine if the detected sign qualifies as one of the particulartraffic control signs in the table or set of data. The imaging systemthus may determine what type of sign has been detected by matching theparameters of the detected sign or object with the expected or specifiedparameters of one of the signs listed in the look-up table or database.

After the sign has been identified as a certain type of sign, furtherprocessing of the sign may commence to determine or read the charactersor information on the face of the sign. The imaging processor 16 mayfurther identify or read the characters on the detected sign viarecognition of the shapes or geometries and arrangement of thecharacters on the sign, such as via utilization of the image processingand/or edge detection discussed above. For example, after the imageprocessor has identified a detected sign as being representative of aspeed limit sign, the image processor may determine what numbers areshown on the sign to determine the speed limit for the zone or areathrough which the vehicle is traveling. The imaging system knows thatthe characters “read” from the sign are for the speed limit (rather thanfor an exit number or a billboard advertisement or the like) based onthe initial analysis of the sign's size/shape/color/location discussedabove. The image processor then may generate an output to cause thedisplay device to display information about the current speed limit asindicated by the detected sign and characters. For example, the displaydevice may display the speed limit to the driver of the vehicle.

Optionally, the image processor 16 may receive an input signal from avehicle speed sensor or sensing means 24, and may display the amount(such as in miles per hour or kilometers per hour or the like) that thevehicle is traveling in excess of (or under) the current speed limit.The speed sensor may comprise any type of sensor or sensing means fordetermining the speed of the vehicle, such as a wheel speed sensor, aglobal positioning system or the like. Optionally, the vehicle speed maybe determined via processing of the images captured by the imagingdevice 14, such as via principles described in U.S. patent applicationSer. No. 10/427,051, filed Apr. 30, 2003, now U.S. Pat. No. 7,038,577,and/or U.S. provisional application Ser. No. 60/638,687, filed Dec. 23,2004, which are hereby incorporated herein by reference.

Optionally, a desirable display may comprise the actual vehicle speedshown at or near or adjacent to or side by side the actual posted anddetected speed limit, in order to provide a visible display of thecurrent speed and the posted or allowed speed limit. It is envisionedthat the display may provide numbers or bars or icons or the like toindicate the vehicle speed and posted speed limit for such a display.The display may adjust the display of the speed and/or the posted speedlimit (such as by increasing the intensity of the display or flashingthe display or the like) if the vehicle speed is above (or below) theposted and detected speed limit by a threshold amount.

Optionally, one or both of the display elements may be highlighted oradjusted in a different manner depending on the driving conditionencountered by the vehicle. For example, when the vehicle speed iswithin the specified threshold/tolerance of the posted speed limit, thedisplay may be set at a particular intensity or color or the like (suchas, for example, a green color), but when the vehicle speed is above thespecified threshold or tolerance, the display may be adjusted to adifferent intensity (such as brighter) or color or the like (such as,for example, a red color). Similarly, when the vehicle speed is belowthe specified threshold/tolerance, the display may be adjusted to adifferent intensity or color or the like (such as, for example, a bluecolor). Other intensities or flashing or color changes or highlightingof one or more display elements may be implemented in response to thedifferent driving/speed conditions encountered by the vehicle, withoutaffecting the scope of the present invention.

Optionally, the image processor may provide an alert or warning to thedriver when the vehicle speed exceeds a threshold amount over (or under)the posted (and recognized) speed limit. For example, the display devicemay flash or adjust the intensity of the displayed speed limit or theimage processor may actuate an audible signaling device 26 to provide anaudible warning, such as a beep or voice warning or the like, when thevehicle speed exceeds (or falls below) the posted and recognized speedlimit by a threshold amount (such as approximately five or ten miles perhour above or below the posted limit or the like). For example, theimaging system may provide a higher pitch audible tone when the vehiclespeed is above the posted speed limit (or at a threshold amount abovethe posted speed limit), and may provide a lower pitch audible tone whenthe vehicle speed is below the posted speed limit (or at a thresholdamount below the posted speed limit). Other alerts or signals may beprovided by the imaging system, such as tactile/haptic type alerts, suchas a rumble or vibration of the seat or steering wheel or the like,without affecting the scope of the present invention. The desiredthreshold amount may be selectively entered by the driver, such as via akey pad, a touch pad, a voice receiver or the like, such that theimaging system may only provide such a warning when it may be desired bythe particular driver of the vehicle. Optionally, it is envisioned thatthe operation of the vehicle may be influenced by the posted anddetected speed limit, such as by a governor or the like that may limitthe maximum speed of the vehicle to within a threshold amount above theposted speed limit.

Optionally, the image processor may provide an alert or warning when thedetected and posted speed limit changes, such as when the vehicle movesfrom one speed zone (such as 55 miles per hour or the like) to anotherspeed zone (such as 35 miles per hour or the like), so as to warn thedriver to slow down (or to speed up if the later zone has a higher speedlimit). For example, when a speed limit is detected that is lower (orhigher) than the previously detected speed limit, the image processormay cause the display device to display the new speed limit, and mayflash or otherwise change or enhance the display to draw the driver'sattention to the display. Optionally, the display device may display anotice that the speed limit has changed, such as “Speed LimitReduced—Slow Down” or the like. Optionally, the image processor mayactuate an audible signaling device to provide a tone or beep or voicemessage to audibly communicate to the driver that the driving conditionshave changed, or may actuate a tactile/haptic signaling device (or othertype of signaling device) to provide a tactile or haptic signal (orother type of signal or alert) to the driver of the vehicle tocommunicate such changes in the driving conditions to the driver of thevehicle.

Optionally, the threshold amount over/under the posted and determinedspeed limit at which the alert is provided may be dynamic and thus maychange depending on the determined speed limit. More particularly, thethreshold amount over a posted speed limit may be greater for speedlimit zones having higher speed limits, such as 55 miles per hour orabove, while the threshold amount may be lower for speed limit zoneshaving lower speed limits, such as 25 miles per hour or 35 miles perhour or less. For example, if the threshold amount is selected to be tenmiles per hour over the speed limit when the speed limit is seventymiles per hour, the imaging system may dynamically adjust or reduce thethreshold amount for lower speed limit zones, so that the thresholdamount may be only, for example, three miles per hour for a 25 miles perhour zone. The imaging system thus may dynamically adapt to the drivingconditions or speed limits or zones encountered by the vehicle, becausewhat may be a safe and reasonable amount over a 65 miles per hour speedlimit (such as five to ten miles per hour) may be much worse or lesssafe if applied to a slower zone, such as a 25 miles per hour zone orthereabouts.

The imaging system may also be operable to detect and recognize and readwarning signs, such as at turns or hills or the like, or may detect andrecognize and read other types of warning signage or the like. Forexample, the imaging system may detect a warning sign that indicatesthat a turn is approaching and that the safe speed of travel around theturn is reduced to a lower speed, such as, for example, 45 miles perhour for a turn located in a 55 miles per hour zone, or such as, forexample, a reduced speed for an exit ramp off of a highway or freeway orthe like. The imaging system may then display the reduced speed limit orreduced recommended speed to alert the driver of the slower speed zoneand/or may then generate a warning signal or alert signal (such as avisible and/or audible and/or tactile/haptic signal) to the driver ifthe current vehicle speed is greater than the reduced or safe or postedspeed (or substantially greater than the posted speed or at or above athreshold amount greater than the posted speed or the like). The drivermay then be alerted to the potentially hazardous condition and mayadjust the speed of the vehicle accordingly.

Optionally, the imaging system may be operable to detect and identify orrecognize other types of signs. For example, the imaging system may beoperable to detect and recognize a railroad crossing sign and to furtherrecognize that the railroad crossing sign is activated (such as bydistinguishing the flashing lights characteristic of a railroad crossingsignal) due to an approaching train. The imaging system could then warnthe driver that the vehicle is approaching a dangerous condition.Additionally, the imaging system may be operable to detect othersignals, such as a school bus stopping signal or a pedestrian roadcrossing signal or the like. Optionally, the imaging system may beoperable to detect road repair or road construction zone signs and mayrecognize such signs to distinguish when the vehicle is entering a roadconstruction zone. The imaging system may display the reduced speed forthe construction zone and/or may provide an alert to the driver of thevehicle that the vehicle is entering a construction zone and that thevehicle speed should be reduced accordingly. The imaging system thus maynot only assist the driver in avoiding a speeding ticket, but mayprovide enhanced safety for the construction workers at the constructionzone.

Optionally, the imaging system of the present invention may beassociated with or cooperatively operable with an adaptive cruisecontrol 28 (FIG. 2) of the vehicle, such that the cruise control speedsetting may be adjusted in response to the imaging system. For example,an adaptive speed control system may reduce the set speed of the vehiclein response to the imaging system (or other forward facing visionsystem) detecting a curve in the road ahead of the vehicle (such as bydetecting and recognizing a warning sign at or before such a curve). Thevehicle speed may be reduced to an appropriate speed for travelingaround the curve without the driver having to manually deactivate thecruise control. For example, the vehicle speed may be reduced to theamount of the reduced or safe limit shown on the warning sign or thelike. The adaptive speed control may then resume the initial speedsetting after the vehicle is through the turn or curve and is againtraveling along a generally straight section of road.

Optionally, the adaptive speed control may adjust the speed setting ofthe vehicle in response to the imaging system recognizing andidentifying a change in speed limit. For example, if the vehicle isinitially traveling at seventy miles per hour in a 65 miles per hourzone, and the imaging system detects a reduced speed limit to 45 milesper hour, the adaptive speed control may reduce the speed setting tofifty miles per hour or thereabouts. The imaging system may also providethe alert or warning to the driver when the speed limit change isdetected, as discussed above. The adaptive speed control may be any typeof adaptive speed control, and may utilize aspects of the controls ofthe types described in U.S. patent application Ser. No. 10/427,051,filed Apr. 30, 2003, now U.S. Pat. No. 7,038,577, and/or U.S.provisional application Ser. No. 60/638,687, filed Dec. 23, 2004, whichare hereby incorporated herein by reference, without affecting the scopeof the present invention.

Although described above as being operable to determine the speed limitor reduced speed posted on a sign detected by the imaging system, theimaging system of the present invention may also process the capturedimages to determine characters on other types of signs as well, such asexit signs or the like. For example, the imaging system may beassociated with or in communication with a navigational system, and maysignal to the driver that the exit sign for a desired exit isapproaching to limit or substantially preclude the possibility that thedriver may miss the desired or targeted exit. The navigational systemmay comprise any type of navigational system, such as the typesdescribed in U.S. Pat. Nos. 6,477,464; 5,924,212; 4,862,594; 4,937,945;5,131,154; 5,255,442 and/or 5,632,092, and/or U.S. patent applicationSer. No. 10/456,599, filed Jun. 6, 2003, now U.S. Pat. No. 7,004,593;Ser. No. 10/287,178, filed Nov. 4, 2002, now U.S. Pat. No. 6,678,614;Ser. No. 10/645,762, filed Aug. 20, 2003, now U.S. Pat. No. 7,167,796;and Ser. No. 10/422,378, filed Apr. 24, 2003, now U.S. Pat. No.6,946,978; and/or PCT Application No. PCT/US03/40611, filed Dec. 19,2003; and/or PCT Application No. PCT/US04/015424, filed May 18, 2004,which are all hereby incorporated herein by reference, without affectingthe scope of the present invention.

Optionally, the imaging system may be operable to utilize data orinformation pertaining to a lane change and/or an exit sign or the like,and an adaptive cruise control system may adjust the speed of thevehicle or the acceleration of the vehicle in response to such lanedivergent information and/or exit ramp information. For example, theimaging system may detect an exit sign along a freeway or the like, andmay detect a lane change by the subject vehicle onto the exit ramp. Theadaptive cruise control system may receive an input that is indicativeof such detections and/or image processing, and may adjust the speed ofthe vehicle accordingly. For example, the adaptive cruise control systemmay decrease the speed of the vehicle and/or may inhibit acceleration ofthe vehicle in response to such detections/image processing, in order tolimit or substantially preclude potentially hazardous conditions wherethe vehicle may accelerate to an unsafe speed on the exit ramp.

As discussed above, the imaging device and/or the display device may bepositioned at or in an interior rearview mirror assembly of the vehicle.For example, the imaging device and/or the display device and/or theimage processor may be positioned within a prismatic mirror assembly,such as a prismatic mirror assembly utilizing aspects described in U.S.Pat. Nos. 6,318,870; 5,327,288; 4,948,242; 4,826,289; 4,436,371 and4,435,042, and PCT Pat. Application No. PCT/US04/015424, filed May 18,2004, which are hereby incorporated herein by reference. Optionally, theprismatic reflective element may comprise a conventional prismaticreflective element or prism or may comprise a prismatic reflectiveelement of the types described in PCT Application No. PCT/US03/29776,filed Sep. 19, 2003; U.S. patent application Ser. No. 10/709,434, filedMay 5, 2004, now U.S. Pat. No. 7,420,756; and Ser. No. 10/993,302, filedNov. 19, 2004, now U.S. Pat. No. 7,338,177, which are all herebyincorporated herein by reference, without affecting the scope of thepresent invention.

Alternately, for example, the interior rearview mirror assembly maycomprise an electro-optic or electrochromic mirror assembly, which mayutilize some of the principles described in commonly assigned U.S. Pat.Nos. 6,690,268; 5,140,455; 5,151,816; 6,178,034; 6,154,306; 6,002,544;5,567,360; 5,525,264; 5,610,756; 5,406,414; 5,253,109; 5,076,673;5,073,012; 5,117,346; 5,724,187; 5,668,663; 5,910,854; 5,142,407 and/or4,712,879, which are hereby incorporated herein by reference, and/or asdescribed in the following publications: N. R. Lynam, “ElectrochromicAutomotive Day/Night Mirrors”, SAE Technical Paper Series 870636 (1987);N. R. Lynam, “Smart Windows for Automobiles”, SAE Technical Paper Series900419 (1990); N. R. Lynam and A. Agrawal, “Automotive Applications ofChromogenic Materials”, Large Area Chromogenics: Materials and Devicesfor Transmittance Control, C. M. Lampert and C. G. Granquist, EDS.,Optical Engineering Press, Wash. (1990), which are hereby incorporatedby reference herein; and/or as described in U.S. patent application Ser.No. 10/054,633, filed Jan. 22, 2002, now U.S. Pat. No. 7,195,381, whichis hereby incorporated herein by reference. The mirror assembly mayinclude one or more other displays, such as the types disclosed in U.S.Pat. Nos. 5,530,240 and/or 6,329,925, which are hereby incorporatedherein by reference, and/or display-on-demand transflective typedisplays, such as the types disclosed in U.S. Pat. Nos. 6,690,268;5,668,663 and/or 5,724,187, and/or in U.S. patent application Ser. No.10/054,633, filed Jan. 22, 2002, now U.S. Pat. No. 7,195,381; PCTApplication No. PCT/US03/29776, filed Sep. 9, 2003; PCT Application No.PCT/US03/35381, filed Nov. 5, 2003; and/or PCT Application No.PCT/US03/40611, filed Dec. 19, 2003, which are all hereby incorporatedherein by reference.

Optionally, the imaging device and/or display device and/or imageprocessor may be positioned, for example, in or at or near an accessorymodule or windshield electronics module or console, such as the typesdescribed in U.S. patent application Ser. No. 10/355,454, filed Jan. 31,2003, now U.S. Pat. No. 6,824,281; and Ser. No. 10/456,599, filed Jun.6, 2003, now U.S. Pat. No. 7,004,593, and/or U.S. Pat. Nos. 6,690,268;6,250,148; 6,341,523; 6,593,565 and 6,326,613, and/or in PCT ApplicationNo. PCT/US03/40611, filed Dec. 19, 2003, which are hereby incorporatedherein by reference). Optionally, the imaging device may be positionedelsewhere in or at the vehicle, such as at or in the headliner of thevehicle or elsewhere at or in the vehicle, without affecting the scopeof the present invention.

Optionally, the accessory module may include other accessories orcircuitry therein, or may be associated with other accessories orcircuitry of the interior rearview mirror assembly and/or of thevehicle. For example, the accessory module or mirror assembly may beassociated with a proximity sensing device or antenna positioned alongthe interior surface of the windshield. The sensing device may detectthe presence of an object, such as a raindrop or water droplets, at theexterior surface of the windshield and, thus, may function as a rainsensing device or rain sensor for sensing precipitation at the exteriorsurface of the windshield. The proximity sensing device may bepositioned at an area of the windshield that is swept by the windshieldwiper to clean the area.

The sensing device or antenna may detect the presence of moisture orprecipitation when rain drops or condensation or the like are within itsrange of detection, and may generate an output signal in response tosuch a detection. The control may process the signals received from thesensing device to determine if an object indicative of rain drops orprecipitation is detected or sensed at the windshield. The control maythen actuate the windshield wipers of the vehicle in response to suchindication. Optionally, the sensing device may sense the presence ofobjects, such as moisture, at the interior surface of the windshield andthe control may process the signals to determine if the detected objectis indicative of moisture at the windshield surface.

The control may actuate or control a blower motor or a control settingof a heating, ventilation and air conditioning (HVAC) system of thevehicle to defog the windshield and/or may close a sunroof or window ofthe vehicle when the control detects moisture on the surface of thewindshield, such as by utilizing aspects of the rain sensors describedin U.S. Pat. Nos. 6,516,664; 6,320,176; 6,353,392; 6,313,454; 6,341,523and 6,250,148; and/or in U.S. patent application Ser. No. 10/355,454,filed Jan. 31, 2003, now U.S. Pat. No. 6,824,281; and Ser. No.10/348,514, filed Jan. 21, 2003, now U.S. Pat. No. 6,968,736, which arehereby incorporated herein by reference. The proximity sensor mayutilize the principles described in U.S. Pat. No. 5,594,222; and/or U.S.patent application Ser. No. 10/956,749, filed Oct. 1, 2004, now U.S.Pat. No. 7,446,924; and/or Ser. No. 10/933,842, filed Sep. 3, 2004, nowU.S. Pat. No. 7,249,860; and/or PCT Application No. PCT/US03/40611,filed Dec. 19, 2003, which are hereby incorporated herein by reference.

Optionally, the proximity sensor may comprise a substantiallytransparent antenna or substantially transparent metallized antenna orsubstantially transparent conductor, such as a wire or wires embedded inthe windshield or a conductive coating (such as indium tin oxide (ITO)or the like) on a window or panel surface, such as the interior surfaceof the windshield. The proximity sensor of the present invention thusmay provide or span or interrogate a larger sensing area withoutobstructing the field of view of the driver or occupant of the vehicle.Optionally, the proximity sensor may comprise multiple sensors orsensing elements or a multi-element sensing array or matrix that isoperable to interrogate the windshield surface over a large area of thewindshield. By interrogating a large area of the windshield, the rainsensing system of the present invention may sample multiple smallsegments of the whole sensing area. Such samplings may enhance thesystem's ability to discern between large raindrops on the windshieldand small raindrops or mist on the windshield and othernon-precipitation items, such as dirt or dust or the like, on thewindshield.

Optionally, the antenna or proximity sensor or sensors or sensingelements may be incorporated into or attached to or associated with awindshield electronics module or accessory module positioned generallyat or against the interior surface of the windshield. For example, thesensing element or elements may be attached to or positioned at ormolded in the wall of the module that opposes and/or engages theinterior surface of the windshield. The sensing element or elements maybe electrically connected to rain sensor or control circuitry within theaccessory module or elsewhere within the vehicle, such as at an interiorrearview mirror assembly or overhead console or instrument panel of thevehicle.

Alternately, the sensing element or elements may be attached to theinterior surface of the windshield, such as via an adhesive, such as viaan adhesive tape such as a double sided adhesive tape or the like. Thesensing element or elements thus may be positioned along the windshieldsurface without having to press the sensing element against thewindshield surface to optically couple the sensing element to thewindshield surface, as is often required in connection with many knownrain sensing imaging devices.

The sensing element or elements of the present invention thus may bereadily attached to the windshield surface, or may be formed on thewindshield surface or may be embedded into the windshield, or may beincorporated into a windshield electronics module or accessory module atthe windshield, without having to press the sensing element against thewindshield surface. The sensing element or elements may be substantiallytransparent or not readily discernible by a driver or occupant of thevehicle, so that the sensing elements may cover and/or interrogate alarge area of the windshield to provide enhanced sensing capabilities,without obstructing the field of view of the driver or occupant of thevehicle. The sensing element or elements may be implemented inconjunction with a rain sensor control that is operable to processsignals from the sensing elements and to control a windshield wiper ofthe vehicle or a blower of the vehicle or an HVAC system of the vehicleor a defogger of the vehicle or a window or sunroof of the vehicle (suchas to close the window or sunroof when rain is detected) or the like, inresponse to the signal processing.

Optionally, the accessory module and/or the interior rearview mirrorassembly may include a forward facing braking indicator that isactuatable in response to a braking of the subject vehicle. The forwardfacing braking indicator may be viewable by a driver or occupant of aleading vehicle and may indicate to the driver or occupant of theleading vehicle that the subject vehicle approaching them is braking.The indicator may be in communication with a brake system of thevehicle, such as to a brake switch at the brake pedal or the like, andthus may indicate when the brakes are applied by the driver of thesubject vehicle. The indicator may be operable in conjunction with thebrake system and/or independently of the brake system (such as inresponse to a deceleration sensor or the like), and may utilize theprinciples described in U.S. Pat. Nos. 6,124,647; 6,291,906 and6,411,204, which are hereby incorporated herein by reference.

The indicator thus alerts the other drivers or people in front of thesubject vehicle that the vehicle is braking and, thus, may be highlyuseful at intersections with two, three or four way stops or the like.The indicator may be at or near or associated with an accessory moduleor windshield electronics module or console or interior rearview mirrorassembly or the like of the vehicle and may be readily viewable anddiscernible by a person outside of and forwardly of the subject vehicle.The control may adjust or modulate the indicator to enhance theviewability or discernibility of the indicator, such as flashing orincreasing the intensity of the indicator, such as in response to rapidor hard braking or the like of the subject vehicle or in response to aproximity or distance sensor detecting that the subject vehicle iswithin a threshold distance of another vehicle and/or is approaching theother vehicle at or above a threshold speed, such as described in U.S.Pat. Nos. 6,124,647; 6,291,906 and 6,411,204, which are herebyincorporated herein by reference.

Optionally, the imaging device may be associated with an accessorycontrol system, such as a headlamp control system or the like. Theimaging device may capture images of the field of view forwardly of thevehicle and the control may process the images and adjust a headlampsetting in response to such processing. Examples of such automaticheadlamp control systems are described in U.S. Pat. Nos. 5,796,094;6,097,023 and 6,559,435, and U.S. patent application Ser. No.10/421,281, filed Apr. 23, 2003, now U.S. Pat. No. 7,004,606.

Optionally, the headlamp control may adjust a direction of the headlampsin response to such image processing. For example, the control mayprocess the captured images to identify headlamps of oncoming vehiclesand/or taillights of leading vehicles and may adjust the downward angleof the headlamps in response to such identification. The headlamps maybe adjusted based on the identification of the headlamps or taillightsand a predetermined or learned knowledge of the location of headlamps ortaillights on vehicles, in order to adjust the headlamps to a desired orappropriate downward angle.

Optionally, the headlamps may be adjusted to compensate for vehicleloading so that the headlamps are directed in a desired directionregardless of the forward pitch or angle of the vehicle. For example, aforward (or rearward) edge of the low headlamp beam in front of thevehicle (such as at the road surface in front of the vehicle) may beidentified and, based on the location of the detected edge or distanceto the detected edge, the control may determine the loading angle orpitch of the vehicle. The control may then raise or lower the headlampangle accordingly, so that the headlamp angle is set to approximatelythe desired direction regardless of the pitch or angle of the vehicle.

Optionally, the control may be operable to process the captured imagesto determine undulations in the road on which the vehicle is travelingand may utilize the processed information to determine the angle of thevehicle and the valleys and peaks in the road. The control may thenadjust the headlamp beam angle and/or direction according to thedetected undulations in the road. The control may be operable inconjunction with a forward facing imaging device and/or a rearwardfacing imaging device and may be operable in conjunction with or may beincorporated in a lane change assist system or lane departure warningsystem or the like, such as the types described in U.S. Pat. Nos.5,929,786 and/or 5,786,772, and/or U.S. patent application Ser. No.10/427,051, filed Apr. 30, 2003, now U.S. Pat. No. 7,038,577; and Ser.No. 10/209,173, filed Jul. 31, 2002, now U.S. Pat. No. 6,882,287, whichare hereby incorporated herein by reference.

Optionally, the control may process the captured images to detectheadlamps of oncoming vehicles and to steer or direct the headlamp beamsat least partially away from the detected oncoming vehicle. For example,the control may determine that an oncoming vehicle is approaching thesubject vehicle in a lane that is to the left of the subject vehicle,and may steer the headlamp beams inboard or to the right to limit orreduce directing the headlamps into the eyes of the driver of theoncoming vehicle. The control thus may steer the headlamp beams inboard(or may steer the outboard or left headlamp beam inboard while keepingthe right or opposite headlamp beam at the initial direction) whenoncoming traffic is detected to limit glare to the drivers of theoncoming vehicles. In situations where the oncoming traffic is locatedto the right of the subject vehicle, the control may steer the headlampbeams (or the right headlamp beam while keeping the left or oppositeheadlamp beam unchanged) inboard or to the left to limit the glare tothe drivers of the oncoming vehicle or vehicles. The steering of theheadlamp beam or beams may be done in conjunction with the switching ofthe beams to a low beam setting, or may be done independently of thehigh/low beam setting of the headlamps.

Optionally, the control may process the captured images and may controlor adjust a louver or filter or the like to direct the headlamp beams ina desired or appropriate direction. For example, a louver or baffle orslats or the like may be positioned in front of the headlamps, and theslats of the louver may be angled and adjusted to adjust the amount oflight that passes through the louver. The louver slats thus may adjustedto an increased angle, such as a downward angle, relative to theheadlamps to reduce the amount of light that passes through the louver(and thus that is visible in front of the vehicle) and thus to reducethe range of the headlamps. The louver control thus controls or adjuststhe visible intensity and range of the headlamps, and may be operable todo this in response to a detection of oncoming traffic or a detection ofleading traffic or any other input or detection, without affecting thescope of the present invention.

Optionally, the control may be operable to process the captured imagesto detect objects in front of the vehicle or forwardly of the vehicleand may control or adjust the display to indicate that an object isdetected. For example, and particularly during nighttime drivingconditions, the control may process the captured images captured by theforward facing imaging device to detect objects of interest that are inthe forward field of view of the imaging device. The imaging device mayutilize night vision principles, and may be operable in connection withan auxiliary light source or infrared radiation source to enhance thenight vision capabilities of the imaging device. When an object ofinterest is detected, the control may process the image to extract theobject data and may determine a distance to and location of the detectedobject relative to the vehicle and the projected path of the vehicle tofurther determine if the object is a threat to the subject vehicle, suchas an animal moving toward or standing in the road ahead of the vehiclebut not yet viewable/discernible to the driver of the vehicle.

If the detected object is also determined to be a threat or danger tothe vehicle, such as a deer at the side of the road or on the road aheadof the vehicle but not yet viewable/discernible by the driver of thevehicle, the control may adjust or actuate or control a display deviceor element to display the detected object to the driver or may otherwisealert the driver of the potentially hazardous condition ahead.Optionally, the control may extract the object data or image data of theobject (without the image data of the rest of the captured image) andmay present the object image to the driver, or may identify the objectand present an icon or indicia or message that indicates to the driverwhat the particular detected object is that is ahead of the vehicle. Thecontrol may control a display at the interior rearview mirror assemblyor at an accessory module or the like, or may control a heads up display(HUD) that is operable to display the object or indicia in front of thedriver and in the driver's field of view, so that the driver is aware ofthe detected object. Preferably, the control may display only thedetected object (such as an image of a detected deer that is extractedfrom the captured image) at a location in the driver's field of viewthat generally or substantially corresponds to the location at which theobject is actually positioned in front of the vehicle. Because the otherimage data is not projected or displayed, the driver is only notified ofor alerted to the particular detected object or objects which thecontrol determines present a hazardous condition (such as in response tothe size of the object, the location of the object, the speed of thevehicle and/or the object, the direction of travel of the vehicle and/orthe object, and/or the like).

In order to properly position the image of the object in the driver'sfield of view, such as via a heads up display, the control may also beoperable in conjunction with an occupant detection system or cabinimaging system or the like that is operable to detect and determine thehead position of the driver of the vehicle. The cabin imaging systemthus may detect the head position of the driver, and the control maydetermine the appropriate location for the object image in the heads updisplay in accordance with the driver's head position.

Optionally, the control may be operable in conjunction with or may beincorporated into a lane departure warning system or the like, and maydetect and identify lane markers along the road lane in front of thevehicle. The imaging device may be operable to detect or capture imagesof the lane markers in situations where the driver may not readily doso, such as in darkened or nighttime conditions or when there is glareon the road surface. The control may identify the lane markers in thecaptured images and may extract (via image processing) the lane markerdata from the captured images or captured image data, and may projectthe lane marker images via a heads up display so that the driver mayview the lane markers in the heads up display, where the lane markerimage in the heads up display in the driver's field of viewsubstantially corresponds to the actual location of the lane markers onthe road surface.

In order to properly position the image of the lane markers in thedriver's field of view, the control may be operable in conjunction withan occupant detection system or cabin imaging system as described aboveto determine the driver's head location. It is further envisioned thatthe control may adjust the display of the lane markers to indicate alane drift or departure by the vehicle. For example, the lane markerimages may be flashed or enhanced, such as by increasing the intensityor changing the color of the lane marker images, when such a lane driftis detected. The driver of the vehicle thus may be alerted to a lanechange or drift or departure by adjusting the heads up display of thelane markers to draw the driver's attention to the lane markers withoutproviding other unnecessary information to the driver. The lanedeparture warning system may utilize the principles described in U.S.Pat. Nos. 5,929,786 and/or 5,786,772, and/or U.S. patent applicationSer. No. 10/427,051, filed Apr. 30, 2003, now U.S. Pat. No. 7,038,577;and Ser. No. 10/209,173, filed Jul. 31, 2002, now U.S. Pat. No.6,882,287; and/or U.S. provisional application Ser. No. 60/638,687,filed Dec. 23, 2004, which are hereby incorporated herein by reference.

Optionally, the imaging device may be selectively operable to provide aforward facing field of view and a rearward facing field of view orcabin viewing field of view. For example, and with reference to FIG. 6,a forward facing imaging device 414 may be positioned within anaccessory module or pod 420 and may be directed generally forwardly toprovide a forward field of view through the windshield 421 of thevehicle. The accessory module 420 may include a movable reflector 438that may be selectively moved relative to the imaging device 414, suchas along the windshield and in front of the imaging device as shown inFIG. 6, to reflect an image of the cabin of the vehicle to the imagingplane or array of the imaging device. The accessory module may include awindow or opening 420 a at the windshield or toward the windshield forreceiving images of the scene forwardly of the vehicle therethrough, andmay also include a window or opening 420 b along a lower or rearwardside or portion of the module for receiving images of the sceneoccurring within the vehicle cabin therethrough. Although shown in FIG.6 as reflecting an image from generally below the module to the imagingdevice, clearly, the angle of the movable reflector may be adjusted orselected to provide a more rearwardly directed field of view, dependingon the application of the imaging system. For example, the angle may beselected to provide a generally rearward field of view for use with abackup aid or rear vision system, or the angle may be selected toreflect images from one side of the module, such as for use with anoccupant detection system or a head position detection system or thelike.

The movable reflector 438 may be selectively moved between a removedposition (as shown in FIG. 6), where the imaging device has a forwardfield of view and is operable to capture images of the scene occurringforwardly of the vehicle (such as for headlamp control, rain sensing,object detection and the like), and a reflecting position (as shown inphantom in FIG. 6), where the imaging device receives the reflectedimage of a rearward view or of the cabin of the vehicle (such as for abackup aid or reverse imaging system or a cabin monitoring system orhead position sensing system or the like). The movable reflector may beslidably moved along a portion of the accessory module or may be pivotedor otherwise moved between the removed position and reflecting position.The movable reflector may be moved between the positions automatically,such as in response to activation of a forward imaging system or a cabinimaging system or a backup aid, or may switch between the positions toprovide the desired or appropriate head location data for use inconjunction with a forward imaging system and display, such as describedabove. Alternately, it is envisioned that the imaging device may beselectively movable to be directed forwardly through the windshield ortoward a stationary reflector for capturing images of the cabin orrearward of the mirror assembly or accessory module, without affectingthe scope of the present invention.

Optionally, the imaging system may be operable to determine thetemperature at the imaging device, in order to determine or approximatethe operating temperature of the imaging device. Although it is known tomonitor the operating temperature of an imaging device in order tomanage or allow for thermal shutdown of the imaging device to avoidoverheating of the device, such systems or devices typically includeseparate temperature sensors positioned at or nearby the imaging sensorto determine the surrounding temperature. According to an aspect of thepresent invention, the imaging device, which comprises an imaging arrayhaving an array of photo-sensing pixels, may be operable to approximatethe operating temperature based on the dark current of some of thepixels of the imaging array. More particularly, one or more pixels ofthe imaging array may be masked so that little or no light reaches thepixel. Because the changes in dark current (the current through thepixel when no light is received by the pixel) is generally proportionateto the changes in temperature of the pixel, a measurement of the darkcurrent, in conjunction with a precalculation and/or relationship of thedark current and temperature, may provide an indication or approximationof the temperature at the pixelated array.

The control of the imaging system thus may be operable to shut down theimaging array sensor or shut down other electronic components of thecontrol system or imaging system in response to the calculated orapproximated or estimated temperature being greater than a predeterminedthreshold that is indicative of a temperature threshold for safe oreffective operation of the imaging device and system. Optionally, thecontrol may be operable to correct or adjust the sensor or system inresponse to detection or calculation of a threshold temperature, inorder to correct or compensate for the increased temperature at theimaging sensor to enhance the performance of the imaging system orcontrol system.

Optionally, the accessory module and/or interior rearview mirrorassembly or system of the vehicle may include a hands free phone system,and thus may include the interface driver, microphone or microphones,user inputs, speech recognition system and/or the like. An example ofsuch a system is described in PCT Application No. PCT/US03/40611, filedDec. 19, 2003, which is hereby incorporated herein by reference. Theaudio signal from the system of the module or mirror assembly ispreferably linked to the radio head, such as to a plug or connector atthe radio head that accepts external audio signals and mute signals. Thesystem thus may mute the audio and effectively take over the speakerswhen the phone is in use. This connection to the vehicle audio or radioor speaker system may utilize a communication link, such as a BLUETOOTH®communication protocol or link or the like. The signals from the mobileor cellular phone to the mirror assembly or accessory module may becommunicated via a BLUETOOTH® link, while the signals from the mirrorassembly or accessory module to the radio head may also be communicatedvia a BLUETOOTH® link. The mirror assembly or accessory module may alsoinclude a display, such as a transflective or display on demand display,to display at least some of the phone information, such as the numberdialed, the incoming number, the status of the call, strength of signal,phone book, messages, and/or the like. Although described as utilizing aBLUETOOTH® communication link or protocol, other communication protocolsor links may be implemented, such as other short/restricted range radiofrequency (RF) or infrared (IR) communication protocol or link.

Optionally, a communication link between an accessory module orwindshield electronics module and the interior rearview mirror assemblymay be provided wirelessly and/or along and/or through the mounting armof the mirror assembly. For example (and as described in U.S. patentapplication Ser. No. 10/456,599, filed Jun. 6, 2003, now U.S. Pat. No.7,004,593, which is hereby incorporated herein by reference), thecommunication link may be via an infrared transmitter and receiver atthe respective module and mirror assembly. Optionally (and as describedin U.S. patent application Ser. No. 10/964,512, filed Oct. 13, 2004, nowU.S. Pat. No. 7,308,341, which is hereby incorporated herein byreference), the communication link may be a two way link with thesignals being communicated along the same wiring. Optionally, themounting arm of the mounting assembly may include a passagewaytherethrough for routing an accessory wiring or the like through the armto provide electrical communication between the circuitry or accessoryof the mirror assembly and the circuitry or accessories or power sourceof the accessory module or of the vehicle. For example, the mountingassembly may utilize principles described in U.S. patent applicationSer. No. 10/032,401, filed Dec. 20, 2001, now U.S. Pat. Publication No.US2002/0088916, published Jul. 11, 2002, now U.S. Pat. No. 6,877,709;and/or U.S. patent application Ser. No. 10/933,842, filed Sep. 3, 2004,now U.S. Pat. No. 7,249,860; and/or PCT Application No.PCT/US2004/015424, filed May 18, 2004; and/or U.S. provisionalapplications, Ser. No. 60/653,787, filed Feb. 17, 2005; Ser. No.60/642,227, filed Jan. 7, 2005; Ser. No. 60/638,250, filed Dec. 21,2004; Ser. No. 60/624,091, filed Nov. 1, 2004, and Ser. No. 60/609,642,filed Sep. 14, 2004, which are all hereby incorporated herein byreference, or may utilize electrical connection principles of the typedescribed in International Publication No. WO 2003/095269, publishedNov. 20, 2003, which is hereby incorporated herein by reference.Optionally, the mounting arm passageway may allow for infrared orvisible light to be transmitted along the tube or arm to communicatesignals to or from the mirror assembly. In such applications, the arm ormounting assembly may include reflectors or mirrored surfaces to guideand reflect the light between the source and receiver, and may adjustthe reflectors to accommodate adjustment of the mirror head assemblyrelative to the mounting base. The mounting arm thus may provide a lightconduit or path or pipe for light signals to be communicated or guidedor directed to provide communication between the accessory module or podand the interior rearview mirror assembly. Other means for providingelectrical power and/or control to the circuitry and/or accessories ofthe mirror assembly may be implemented without affecting the scope ofthe present invention.

Optionally, the vehicle or the rearview mirror assembly or accessorymodule of the vehicle may include a communication system or interfacesystem that is operable to communicate with a remote or external controlor base or center of a telematic system, such as ONSTAR®, TELEAID™,RESCU® or the like, or with any other remote computerized server ordatabase or information provider or the like. The data captured by animaging device of the vehicle (such as a rearward facing imaging deviceor a cabin monitoring imaging device or a forward facing imaging deviceor another vehicle-based imaging device or camera) may be communicatedto the communication system (the communication system may be at thecamera device or the signals may be communicated to the communicationsystem remote from the camera, such as via vehicle wiring or via a localwireless communication or the like), whereby the communication systemmay communicate the image data to the external control of the telematicsystem. The image data may be processed by the processor at the externalcontrol and a signal indicative of such image processing may becommunicated from the external control to the communication system ofthe vehicle, where the appropriate information may be displayed orotherwise communicated or conveyed to the driver or occupant of thevehicle.

In some known imaging systems for vehicles, image data is communicatedfrom the vehicle camera to a microprocessor in the vehicle where theimage data is processed. Such a system typically requires connection ofthe camera and microprocessor and display or alert device via wires orlocal wireless connections and requires in vehicle processing andconnections. Such a system is typically not conducive for sharinginformation gathered from the image processing with other systems ordevices or vehicles.

The communication system of the present invention receives the imagedata and uploads the image data to the external control for processing.Optionally, the vehicle communication system may conduct a datacompression routine to compress the image data prior to uploading thedata to the external control. For example, the vehicle communicationsystem may compress the data and upload the compressed data using“burst” technology (where compressed data are transmitted orcommunicated to a satellite or the like in short (such as, for example,about twelve milliseconds or thereabouts) signals or bursts) to conveylarge amounts of data to the external control. The external control maythen process the image data and extract the desired or relevantinformation from the image data and may communicate a signal back to thevehicle that is indicative of the extracted information.

The communication system and telematic system of the present inventionthus may harness the processing power of the external control, which maybe substantially greater than the processing power of a vehicle-basedmicroprocessor. The external control may receive the image data and mayrecognize that the data is being communicated from a particular vehicle.In applications where the vehicle includes a global positioning system(GPS), the external control may receive and process the image data andmay receive an input that is indicative of the vehicle location. Theexternal control thus may process the image data and location data toextract information and provide an output that may be relevant to thelocation of the vehicle. For example, the external control may processthe image data and may determine the speed limit signage information inthe appropriate units based on the vehicle location, such as describedabove.

The external control may also receive location data from other vehiclesand thus may know the location of other vehicles relative to the subjectvehicle. For example, if the vehicle-based imaging device is for anadaptive cruise control system, the external control may receive theforward viewing image data and may receive data indicative of thevehicle location. The external control may also receive location datafrom other vehicles and thus may know the relative location andmovements of other vehicles. If, for example, the subject vehicle isapproaching a curve in the road and another vehicle is approaching inthe opposite direction from around the curve, the external control maycommunicate a signal to the subject vehicle that is indicative of thelocation of the other vehicle. For example, the external control mayprovide a signal to the vehicle whereby an alert or warning or displaydevice of the vehicle operates to alert or warn the driver of thesubject vehicle as to the location of the approaching vehicle, in orderto reduce or avoid vehicle collisions.

Optionally, the imaging system of the vehicle may be associated with anadaptive front lighting (AFL) system. The imaging system may also beassociated with a lane departure warning system or side object detectionsystem or lane change assist system or the like. The imaging device ofthe imaging system may be a forward facing imaging device or camera thatis operable to capture images of a forward field of view. The control ormicroprocessor (or external control of a telematic system or the like)may process the image data to identify lane markers and other objects ofinterest in the forward field of view, such as by utilizing theprinciples described in U.S. Pat. No. 5,929,786, and/or U.S. patentapplication Ser. No. 10/427,051, filed Apr. 30, 2003, now U.S. Pat. No.7,038,577; and/or Ser. No. 10/209,173, filed Jul. 31, 2002, now U.S.Pat. No. 6,882,287, and/or U.S. provisional application Ser. No.60/638,687, filed Dec. 23, 2004, which are hereby incorporated herein byreference.

For example, the lane departure warning system may process the imagedata to detect the lane markers along the road surface in front of thevehicle. The lane departure warning system may detect a curvature in theroad as the lane markers (or other characteristics, such as a curb orshoulder of the road) curve in front of the vehicle (such as byutilizing principles described in U.S. provisional application Ser. No.60/638,687, filed Dec. 23, 2004, which is hereby incorporated herein byreference). Such road curvature information extracted from the imagedata may be used as an input or feed signal to the headlamp controlsystem, which may adjust or control the headlamps to direct the headlampbeam toward one side or the other of the vehicle, in order to generallyfollow the curve of the road in front of the vehicle and, thus, togenerally follow the anticipated path of the vehicle.

Typically, a lane departure warning system is interested in and mayprincipally monitor the near field of view of the imaging device, suchas, for example, about ten to twenty feet in front of the vehicle, whilean intelligent headlamp control system and/or an adaptive front lightingsystem may principally monitor a further or far field of view of theimaging device. The processor thus may process different areas of thecaptured image data for the different applications. For example, theprocessor may process the captured image data in a frame-by-framemanner, and may process different areas of the image to extractdifferent information for some of the frames (such as by utilizing theprinciples described in U.S. provisional applications, Ser. No.60/630,061, filed Nov. 22, 2004; Ser. No. 60/628,709, filed Nov. 17,2004; Ser. No. 60/614,644, filed Sep. 30, 2004; and/or Ser. No.60/618,686, filed Oct. 14, 2004, which are hereby incorporated herein byreference).

Thus, if the imaging device captures frames at a rate of about thirtyframes per second (or other frame rate depending on the particularapplication and system capabilities and the like), the processor mayprocess different frames for different functions or systems or theprocessor may selectively process a given frame or frames for more thanone functionality or feature. For example, every third frame may beprocessed for the lane departure warning system, while every fifth framemay be processed for the adaptive front lighting system, while everysecond frame may be processed for the intelligent headlamp controlsystem. Alternatively, any one frame or sets of frames may be processedfor intelligent headlamp control only, while other frames or sets offrames may be processed for lane departure warning and/or adaptive frontlighting. The microprocessor thus may process different portions orareas of the image data for different functions or systems of thevehicle. The less relevant image data from the particular sets of framesthus may be processed less by the microprocessor so that themicroprocessor has reduced or focused processing of the image data setsthat is focused on the particular area of the image data that isrelevant to the particular system or function for that particular frameor set of frames. Optionally, different processors may process the imagedata or may process different frames of image data captured by theimaging device or camera. Optionally, the reduced processed frame datamay accumulate over several frames to provide a history and/orcontent/background for a given functionality, such as for an adaptivefront lighting system.

Thus, a single camera and optical system can provide at least triplefunctionality, such as intelligent headlamp control, lane departurewarning, and adaptive front lighting. For example, the image datacaptured by a single forward facing camera and associated lens andoptical system can be processed and the information extracted can beused to control the headlamps on/off or high beam/low beam settings, todetect and monitor lane markers, and to provide an input or feed to aheadlamp controller that may adjust or redirect the headlamp beam for anadaptive front lighting system.

Desirably, the system may intelligently process the image data andharness the processing power and frame rate to provide enhanced dynamicprocessing of image data depending on the particular lightingconditions. For example, the system may bias the processing towardextracting information from the image data for the headlamp control whenthe ambient lighting conditions are reduced, such as at nighttime, andmay bias the processing toward extracting information from the imagedata for the lane departure warning system when the ambient lightingconditions are increased, such as during daytime or other conditionswhen it is less likely that headlamp control is needed. The processor orimaging system thus provides dynamic processing of the captured imagedata to enhance the performance of the associated headlamp controlfunction, adaptive front lighting function and lane departure warningfunction.

Optionally, a rearview mirror and/or accessory module or windshieldelectronics module of a vehicle may include or may be associated with astorage medium for storing digital data or the like. For example, themirror or module may include circuitry or accessories to record data(such as music from an iPod or MP3 player or the like) to a memory cardand/or disc or other storage medium, such as a mini hard drive, or thelike. For example, the rearview mirror assembly or accessory module mayinclude a hard disc drive (HDD) electronic mass storage device, such asa HDD microdrive, such as a one-inch (or smaller) HDD, such as the typesdeveloped by Hitachi Global Storage Technologies, Inc. (HGST) of theUnited States, Hoya Corp. of Japan, and Seagate Technology LLC, and suchas described in U.S. patent application Ser. No. 10/933,842, filed Sep.3, 2004, now U.S. Pat. No. 7,249,860, which is hereby incorporatedherein by reference. The data that is stored in the storage medium maythen be “played” by the system and streamed through the speakers of thevehicle to play the music or playback the recording or the like that isstored in the storage medium. Optionally, the memory or storage mediummay be removed from the mirror or accessory module and plugged into orconnected to the iPod or MP3 player or the like (and vice versa), inorder to playback the music or information stored on the storage mediumwith the different playback devices.

Optionally, the driver or other occupant of the vehicle may bring his orher digital audio player (such as an iPod or similar MP3 or otherdevice) and dock at an interior mirror location (such as at a videoslide-out mirror) or at a windshield electronics module (WEM) location(such as is disclosed, for example, in U.S. Pat. Nos. 6,428,172;6,501,387 and 6,329,925, which are hereby incorporated herein byreference). Information relating to the audio device (such as tracknumber or song title or index or the like) may be displayed at theinterior mirror assembly (such as using display-on-demand transflectivemirror reflector technology as described herein), or may be displayed ona video slide-out mirror screen (such as disclosed in PCT ApplicationNo. PCT/US03/40611, filed Dec. 19, 2003, and/or U.S. provisionalapplication Ser. No. 60/630,061, filed Nov. 22, 2004, which are herebyincorporated herein by reference) or may be displayed at a WEM. Also,controls to operate the consumer portable device, such as an iPod or thelike, may be included at the interior mirror assembly and/or at a WEM.While docking has the added advantage of providing electrical currentfrom the vehicle battery/ignition system to recharge the portabledevice, such as an iPod or similar MP3 player, the iPod deviceoptionally need not dock and can be in wireless communication with theinterior mirror and/or WEM via a short range wireless communicationprotocol, such as BLUETOOTH® or the like. Of course, if desired, wiredconnection can also be used.

Optionally, a docking station can be provided other than at the interiormirror or WEM. For example, an iPod or a similar audio device or a videoplayback device (such as a DVD player) can dock at a docking cradlelocated between the front seats and can be in wireless communication(such as via BLUETOOTH®) and/or optionally can be in wired communicationwith the interior mirror and/or WEM, where the aforementioned displaysand/or controls may be readily available to the driver or other occupantof the vehicle. The music or other audio or data files stored on theiPod or similar MP3 player or data storage device may be played via theaudio system of the vehicle, and the driver or other front seat occupantcan readily access the displays/controls at the interior mirror or WEMlocation.

Optionally, the data may be automatically recorded and stored on thestorage medium incorporated into an interior mirror assembly and/or aWEM and/or an exterior mirror assembly or may be selectively stored onthe storage medium. For example, a user may connect or plug in theiriPod or MP3 player or cellular telephone or portable telephone or thelike into a receiver or socket (such as at an interior mirror or at aWEM) and the data may be transferred or streamed onto the storage mediumof that vehicular location so that the recording may be played backthrough the vehicle speakers of the vehicle audio system. Optionally,the data transfer may be accomplished wirelessly, such as via an IRand/or an RF wireless link. Optionally, the user may selectively recordinformation or music from radio signals (such as signals broadcast to anAM/FM radio of the vehicle or to an XM satellite radio or the like), orthe user may selectively record information or music from wirelessINTERNET signals or the like (such as from a music download website orthe like) where the transmitted or broadcast information may betransferred or streamed to the storage medium or disc or the like of themirror assembly or accessory module of the vehicle. Optionally, thestored data or information or music may be transferred or streamed fromthe storage medium of the mirror or WEM to a portable device, such as aniPod or MP3 player or cellular telephone or portable telephone or thelike for playback at a different time and remote from the vehicle.

Optionally, the recording or playback system of the vehicle may be voiceactivated, such that a user may provide a voice command to record orplayback a particular track. For example, a user may select a specifictrack or tracks stored on a storage medium (such as the storage mediumof the mirror or accessory module or the like or a storage medium of anMP3 player or iPod device or the like), and the track or tracks may beplayed accordingly. The system of the present invention thus may providefor voice activation of an MP3 player, such as via, preferably, amicrophone or microphones located at the interior mirror assembly or ata WEM or the like, when the player is plugged into or connected to or incommunication with the recording and/or playback system of the presentinvention.

It is further envisioned that the recording and/or playback system ofthe present invention may provide delayed playback of a recording thatis made generally at the same time that it is being played back. Forexample, it may be desirable to remove commercial content from a radio(such as satellite radio or XM radio) transmission, since some satelliteradio transmissions or broadcasts or signals or outputs now may includecommercial content. If desired, the recording and/or playback system ofthe present invention could selectively record a transmission and beginplaying the transmission back with a time delay. While the system playsthe delayed output, the system may identify and remove commercialcontent (or other undesirable content) and continue playing the outputwithout interruption. The user thus may select a record and play modeand hear substantially continuous output without the commercial contentor otherwise undesired content.

Although shown and described as being positioned so as to have a forwardfield of view, it is also envisioned that the imaging device may bedirected to have a field of view generally rearwardly or sidewardly ofthe vehicle to capture images of a rearward or sideward scene, withoutaffecting the scope of the present invention. For example, the imagingdevice may be positioned at a rearward portion of the vehicle and/or maybe used in connection with a rear vision system or the like, such as thetypes described in U.S. Pat. Nos. 5,550,677; 5,760,962; 5,670,935;6,201,642 and/or 6,717,610, and/or in U.S. patent application Ser. No.10/010,862, filed Dec. 6, 2001, now U.S. Pat. No. 6,757,109; Ser. No.10/418,486, filed Apr. 18, 2003, now U.S. Pat. No. 7,005,974, which arehereby incorporated herein by reference.

Optionally, the imaging device may be positioned at or near or at leastpartially within a door handle of a side door of the vehicle. Theimaging device thus may provide a sideward field of view, such as for aside object detection system or lane change assist system or for asecurity system or the like.

The imaging device of the imaging system of the present invention thusis operable to capture multiple images or frames of the scene as thevehicle travels along the road, and may detect and recognize variousstreet and/or traffic signs via processing of the captured images. If adetected sign is determined to be a speed limit sign or other trafficcontrol sign of interest (such as a warning sign or the like), theimaging system may be operable to further process the images todetermine or recognize the speed limit numbers on a speed limit sign andto provide an alert or warning signal to the driver of the vehicle ifthe vehicle exceeds the posted and recognized speed limit by apredetermined amount. The imaging system may have an interface (such asa user actuatable input or button, a voice receiver, a touch screenand/or the like) that would set a personal threshold for over-speedwarning. The imaging device and/or imaging system may be multi-taskingand, thus, may be operable to detect headlamps and taillights and/orprecipitation and/or objects and/or the like for or in connection withother accessories or systems, such as a headlamp control system, aprecipitation sensor system, an adaptive speed control system, a lanedeparture warning system, a traffic lane control system and/or the like.

Changes and modifications in the specifically described embodiments canbe carried out without departing from the principles of the presentinvention, which is intended to be limited only by the scope of theappended claims as interpreted according to the principles of patentlaw.

What is claimed is:
 1. A vision system for a vehicle, said vision systemcomprising: a forward viewing camera disposed at a windshield of avehicle equipped with said vision system; wherein said forward viewingcamera views forward through the windshield of the equipped vehicle andis operable to capture image data; said forward viewing cameracomprising an imaging array sensor and a lens; an image processoroperable to process image data captured by said forward viewing camera;wherein, responsive at least in part to processing by said imageprocessor of image data captured by said forward viewing camera, atleast one road characteristic of a road along which the equipped vehicleis traveling is detected; wherein, responsive at least in part toprocessing by said image processor of image data captured by saidforward viewing camera, other vehicles exterior the equipped vehicle aredetected; wherein, responsive at least in part to processing by saidimage processor of image data captured by said forward viewing camera,road curvature of the road along which the equipped vehicle is travelingis determined; and wherein data derived from image data captured by saidforward viewing camera is wirelessly communicated from the equippedvehicle to a remote data receiver located remote from the equippedvehicle.
 2. The vision system of claim 1, wherein said at least one roadcharacteristic comprises a curb.
 3. The vision system of claim 2,wherein said at least one road characteristic comprises a road shoulder.4. The vision system of claim 1, wherein data derived from capturedimage data received from the equipped vehicle is processed at saidremote data receiver to extract information relevant to the geographiclocation at which the equipped vehicle is located when wirelesslytransmitting the data to said remote data receiver.
 5. The vision systemof claim 4, wherein said remote data receiver receives location datafrom other vehicles.
 6. The vision system of claim 5, wherein, based atleast in part on received location data, movement of other vehiclesrelative to the equipped vehicle is determined.
 7. The vision system ofclaim 6, wherein, responsive at least in part to said determination ofmovement of other vehicles relative to the equipped vehicle, said remotedata receiver communicates to the equipped vehicle that another vehicleis approaching the equipped vehicle from a direction opposite to thedirection of travel of the equipped vehicle.
 8. The vision system ofclaim 7, wherein, responsive at least in part to said determination ofmovement of other vehicles relative to the equipped vehicle, a speedcontrol system of the equipped vehicle is controlled.
 9. The visionsystem of claim 8, wherein said speed control system comprises anadaptive cruise control system of the equipped vehicle.
 10. The visionsystem of claim 6, wherein, responsive at least in part to saiddetermination of movement of other vehicles relative to the equippedvehicle, a lighting system of the equipped vehicle is controlled. 11.The vision system of claim 6, wherein, responsive at least in part tosaid determination of movement of other vehicles relative to theequipped vehicle, a speed control system of the equipped vehicle isdynamically controlled.
 12. The vision system of claim 1, wherein,responsive at least in part to processing by said image processor ofimage data captured by said forward viewing camera, speed of theequipped vehicle is reduced responsive to approach by the equippedvehicle to a road curve of the road along which the equipped vehicle istraveling.
 13. The vision system of claim 12, wherein, upon the equippedvehicle having traveled around the curve, speed of the equipped vehicleautomatically increases.
 14. The vision system of claim 1, wherein theroad along which the equipped vehicle is traveling comprises a freeway,and wherein, responsive at least in part to processing by said imageprocessor of image data captured by said forward viewing cameradetecting an exit sign announcing an exit ramp ahead off the freeway theequipped vehicle is traveling on, at least one of (i) speed of theequipped vehicle and (ii) acceleration of the equipped vehicle isadjusted so that the equipped vehicle enters the exit ramp at a safespeed.
 15. The vision system of claim 1, wherein the road along whichthe equipped vehicle is traveling comprises a freeway, and wherein,responsive at least in part to processing by said image processor ofimage data captured by said forward viewing camera detecting an exitsign announcing an exit ramp ahead off the freeway the equipped vehicleis traveling on and responsive at least in part to determiningcharacters on the detected exit sign, a driver of the equipped vehicleis alerted that a desired exit ramp off the freeway is ahead.
 16. Thevision system of claim 15, wherein, responsive at least in part toprocessing by said image processor of image data captured by saidforward viewing camera, characters on the detected exit sign aredetermined.
 17. A vision system for a vehicle, said vision systemcomprising: a forward viewing camera disposed at a windshield of avehicle equipped with said vision system; wherein said forward viewingcamera views forward through the windshield of the equipped vehicle andis operable to capture image data; said forward viewing cameracomprising an imaging array sensor and a lens; an image processoroperable to process image data captured by said forward viewing camera;wherein, responsive at least in part to processing by said imageprocessor of image data captured by said forward viewing camera, atleast one road characteristic of a road along which the equipped vehicleis traveling is detected; wherein, responsive at least in part toprocessing by said image processor of image data captured by saidforward viewing camera, other vehicles exterior the equipped vehicle aredetected; wherein, responsive at least in part to processing by saidimage processor of image data captured by said forward viewing camera,road curvature of the road along which the equipped vehicle is travelingis determined; wherein the road along which the equipped vehicle istraveling comprises a freeway; and wherein, responsive at least in partto processing by said image processor of image data captured by saidforward viewing camera detecting an exit sign announcing an exit rampahead off the freeway the equipped vehicle is traveling on, speed of theequipped vehicle is adjusted so that the equipped vehicle enters theexit ramp at a safe speed.
 18. The vision system of claim 17, wherein,responsive at least in part to processing by said image processor ofimage data captured by said forward viewing camera, characters on thedetected exit sign are determined.
 19. The vision system of claim 18,wherein, responsive to determination of characters on the detected exitsign by processing by said image processor of captured image data, adriver of the equipped vehicle is alerted that a desired exit ramp offthe freeway is ahead.
 20. The vision system of claim 17, wherein,responsive at least in part to processing by said image processor ofimage data captured by said forward viewing camera detecting the exitsign announcing an exit ramp ahead off the freeway the equipped vehicleis traveling on, speed of the equipped vehicle and acceleration of theequipped vehicle are adjusted so that the equipped vehicle enters theexit ramp at a safe speed.
 21. The vision system of claim 17, whereindata derived from image data captured by said forward viewing camera iswirelessly communicated from the equipped vehicle to a remote datareceiver located remote from the equipped vehicle, and wherein dataderived from captured image data received from the equipped vehicle isprocessed at said remote data receiver to extract information relevantto the geographic location at which the equipped vehicle is located whenwirelessly transmitting the data to said remote data receiver.
 22. Thevision system of claim 21, wherein said remote data receiver receiveslocation data from other vehicles, and wherein, based at least in parton received location data, movement of other vehicles relative to theequipped vehicle is determined.
 23. The vision system of claim 22,wherein, responsive at least in part to said determination of movementof other vehicles relative to the equipped vehicle, a speed controlsystem of the equipped vehicle is controlled.
 24. The vision system ofclaim 23, wherein said speed control system of the equipped vehicle isdynamically controlled.
 25. The vision system of claim 24, wherein saidspeed control system comprises an adaptive cruise control system of theequipped vehicle.
 26. The vision system of claim 24, wherein said atleast one road characteristic comprises a road shoulder.
 27. A visionsystem for a vehicle, said vision system comprising: a forward viewingcamera disposed at a windshield of a vehicle equipped with said visionsystem; wherein said forward viewing camera views forward through thewindshield of the equipped vehicle and is operable to capture imagedata; said forward viewing camera comprising an imaging array sensor anda lens; an image processor operable to process image data captured bysaid forward viewing camera; wherein, responsive at least in part toprocessing by said image processor of image data captured by saidforward viewing camera, at least one road characteristic of a road alongwhich the equipped vehicle is traveling is detected; wherein, responsiveat least in part to processing by said image processor of image datacaptured by said forward viewing camera, other vehicles exterior theequipped vehicle are detected; wherein, responsive at least in part toprocessing by said image processor of image data captured by saidforward viewing camera, road curvature of the road along which theequipped vehicle is traveling is determined; and wherein, responsive atleast in part to processing by said image processor of image datacaptured by said forward viewing camera, speed of the equipped vehicleis reduced responsive to approach by the equipped vehicle to a roadcurve of the road along which the equipped vehicle is traveling.
 28. Thevision system of claim 27, wherein the road along which the equippedvehicle is traveling comprises a freeway, and wherein, responsive atleast in part to processing by said image processor of image datacaptured by said forward viewing camera detecting an exit signannouncing an exit ramp ahead off the freeway the equipped vehicle istraveling on, speed of the equipped vehicle is adjusted so that theequipped vehicle enters the exit ramp at a safe speed, and wherein,responsive at least in part to processing by said image processor ofimage data captured by said forward viewing camera, characters on thedetected exit sign are determined.
 29. The vision system of claim 28,wherein, responsive to determination of characters on the detected exitsign by processing by said image processor of captured image data, adriver of the equipped vehicle is alerted that a desired exit ramp offthe freeway is ahead.
 30. The vision system of claim 28, wherein,responsive to determination of characters on the detected exit sign byprocessing by said image processor of captured image data, speed of theequipped vehicle and acceleration of the equipped vehicle are adjustedso that the equipped vehicle enters the exit ramp at a safe speed. 31.The vision system of claim 27, wherein data derived from image datacaptured by said forward viewing camera is wirelessly communicated fromthe equipped vehicle to a remote data receiver located remote from theequipped vehicle, and wherein data derived from captured image datareceived from the equipped vehicle is processed at said remote datareceiver to extract information relevant to the geographic location atwhich the equipped vehicle is located when wirelessly transmitting thedata to said remote data receiver.
 32. The vision system of claim 31,wherein said remote data receiver receives location data from othervehicles, and wherein, based at least in part on received location data,movement of other vehicles relative to the equipped vehicle isdetermined.
 33. The vision system of claim 32, wherein, responsive atleast in part to said determination of movement of other vehiclesrelative to the equipped vehicle, a speed control system of the equippedvehicle is controlled.
 34. The vision system of claim 33, wherein saidspeed control system of the equipped vehicle is dynamically controlled.35. The vision system of claim 27, wherein said at least one roadcharacteristic comprises a curb.
 36. The vision system of claim 27,wherein said at least one road characteristic comprises a road shoulder.